Invested in Diabetes: Comparing Patient-Centered Outcomes of Standardized Versus Patient-Driven Diabetes Shared Medical Appointments (PCORI IHS-1609-36322)
Phase: NA
Conditions: Diabetes Mellitus, Type 2
Sponsor: University of Colorado, Denver

Comparative Effectiveness of Diabetes Shared Medical 
Appointment Models ("Invested in Diabetes") Protocol 
and Statistical Analysis Plan Cover Page 
Version date 8/24/2021 
PI: Bethany M Kwan 
IRB: Colorado Multiple Institutes Review Board # 
17-2377 NCT# 03590041 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
1 
Comparing Patient-Centered Outcomes of Standardized versus  
Patient-Driven Diabetes Shared Medical Appointments 
 
RESEARCH STRATEGY  
 
A. Background  
Burden of diabetes 
Diabetes is among the most prevalent chronic diseases in the United States, such that as of 2012 an estimated 12-
14% of the US population had diagnosed or undiagnosed diabetes.1 Type II diabetes (T2DM) is the most common form of 
diabetes among adults.2 Diabetes can be controlled with appropriate diet and physical activity, and oral and injectable 
medications.3 The goal is to maintain glycosylated hemoglobin (hemoglobin A1c, or HbA1c) under 7% (or under 8% for 
those age 45-64 with complications), while over 9% is considered poor control.4  Yet as many as 49% of adults with 
diabetes do not meet targets for glycemic control.5 Poorly controlled diabetes is associated with a variety of poor health 
outcomes and disability, including neuropathy, retinopathy, nephropathy, cardiovascular disease, and premature death.6 
Despite recent decreases in rates of certain complications, diabetes remains a considerable source of disability and cost 
to the health care system.7 The burden of diabetes is great, both in terms of patient out-of-pocket health care costs 8 
and poor quality of life, especially among those with complications.9 Patients with T2DM and co-occurring mental illness 
accrue significantly higher costs and experience worse outcomes than those without.10,11  
Patients with diabetes must engage in daily self-care activities including blood glucose monitoring, following dietary 
recommendations, regular physical activity, and medication adherence (including insulin management in those who are 
insulin-dependent). Diabetes self-management is challenging, especially among low-income populations12, and many 
patients experience diabetes distress, the sense of being overwhelmed with managing diabetes.13 Diabetes distress 
stems from the regimen, interpersonal, emotional, and healthcare navigation burden associated with managing 
diabetes, and interferes with self-care and glycemic control.14  As projections indicate the prevalence of diabetes may 
reach 25-28% of the US population by the year 2050,6 we must continue to improve diabetes care. 
Improving care for patients with T2DM 
Standards of care for T2DM currently include medication management, smoking cessation, and diet and physical 
activity counseling.  According to the American Diabetes Association’s 2015 position statement, care for patients with 
T2DM should include antiglycemic therapy and cardiovascular risks reduction through weight loss, blood pressure 
reduction, and smoking cessation.15 Wagner’s chronic care model (CCM) has informed how care should be delivered for 
T2DM patients to help achieve these goals.16-18 The CCM emphasizes whole person care, addressing physical, mental 
health and psychosocial needs.19 Evidence shows patients with diabetes benefit from models of care based on the 
CCM20, including comprehensive diabetes self-management education (DSME) and self-management support (SMS) in 
primary care.21,22  Notably, SMS can decrease the burden of diabetes and improve diabetes distress.23 Shared medical 
appointments (SMAs) can help practices efficiently and effectively provide DSME and SMS , consistent with the CCM. 24  
Shared medical appointments (SMAs) 
SMAs are “groups of patients meeting over time for comprehensive care, usually involving a practitioner with 
prescribing privileges, for a defining chronic condition or health care state”.25 SMAs for chronic disease management are 
increasingly employed in medical clinics26,27 and endorsed by entities such as the American Academy of Family 
Physicians28 as effective for improving self-management behaviors, satisfaction, and health outcomes.  SMAs have been 
successfully used as parts of SMS programs for diabetes and other chronic illnesses in diverse and low resource settings 
such as community health centers.24 A 2014 systematic review and meta-analysis showed diabetes SMAs lead to 
significantly greater improvements in HbA1c and blood pressure compared to usual care.25 However, there was 
heterogeneity in these effects, suggesting some SMA models may be more effective than others. It is not known which 
SMA features are most effective for improving outcomes important to patients and practices.  

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
2 
According to patients and practice stakeholder engaged in 
our research, an ideal SMA model may have the features 
highlighted in Table 1, including 1) a patient-driven curriculum, 
from which patients choose from a range of diabetes and mental 
health topics; and 2) a multidisciplinary care team with diabetes 
peer mentors, health educators, medical providers and 
behavioral health providers. These features correspond to a 
model involving “professionally-led group visits with peer 
exchange and a team of providers”29, and which we refer to as 
patient-driven SMAs. Evidence supports each of these 
components in diabetes SMAs and for diabetes care in general, 
including the use of multidisciplinary care teams30,31, peer 
support32-34, and allowing patients to drive topic selection30, but no comparative 
effectiveness trials comparing patient-driven SMAs to standardized SMA models (patients 
do not select topics nor are there multidisciplinary care team members) .  
 Methodology Standard RQ-1: Gaps in Evidence  
SMAs are carried out in a variety of ways, and it is unclear which components lead to greater patient participation 
and health outcomes. SMAs vary in terms of the makeup of the care provider team, the clinical and educational 
strategies used, the topics addressed and whether patients drive the selection of topics, and the outcomes evaluated.  
Also, there is little evidence on the effects of SMAs on patient-centered outcomes, such as diabetes distress. Thus, the 
proposed study will compare the effects of a standardized diabetes SMA model to the patient-driven model involving 
multidisciplinary care teams and patient-driven content on patient-centered outcomes– these are critical evidence gaps 
for SMAs and priority features for our stakeholders.  
 
B. Significance  
Improving Primary Care Services for Patients with Diabetes 
The CCM is considered an evidence-based and patient-centered model of 
care.35 Patient-centered care often involves individually tailored services that 
meet the needs and preferences of patients, such that more intensive services are recommended to patients at times 
when they are most likely to engage in and benefit from those services.15 With respect to diabetes SMAs, the proposed 
study is designed to test the extent to which – and for whom – more patient-centered SMA models (ensuring the right 
resources for the right patient at the right time) are more effective at engaging patients and improving patient-centered 
outcomes than more standardized models (same intervention for all). Given the importance placed on patient-centered 
approaches to diabetes care36 and the need to efficiently provide care to increasing numbers of patients with diabetes, 
practices need to know the extent to which patient-driven SMAs are more effective and engaging than standardized 
SMAs (see Letters of Support). Practices need this information to justify widespread uptake of enhanced models of care. 
Stakeholder engagement: Selecting questions that affect outcomes of interest to patients and other stakeholders 
Aligning multiple stakeholder needs to design and test care models can benefit from joint engagement strategies.37 
We used the Boot Camp Translation (BCT) approach to jointly engage patients with diabetes, caregivers, and primary 
care providers and staff in translating evidence about SMS into a mutually-desirable diabetes care model.38,39 In BCT, 
stakeholders come together for an intensive review of evidence and facilitated discussion over 6-8 months. In our BCT, 9 
patients with diabetes and three family members and 6 providers and staff members from a local federally qualified 
health center (FQHC) participated. When discussing how to connect patients to evidence-based services, patients 
repeatedly voiced the importance of “meeting people where they’re at” – a commonly-used phrase by our stakeholders 
meaning offering services tailored to individual patient needs. Specifically, diabetes care should address the fact that not 
all patients feel as though they can readily absorb and apply new information about their condition and how to manage 
it – this is health literacy, a known correlate of diabetes control.29,40  
Table 1. SMA FEATURES OF INTEREST TO STAKEHOLDERS 
FEATURES 
EXAMPLES 
Multidisciplinary 
care team 
SMAs have “guest speakers” representing 
clinical and behavioral health specialties 
Peer support 
Peer mentors co-facilitate SMAs and work 
with patients one-on-one 
Whole-person 
orientation  
SMA curriculum includes health behavior 
change and mental health content 
Patient-driven 
content and 
structure 
Modular curriculum with topics selected by 
SMA participants and patient-driven care 
team and family involvement 
Focus on patient-
centered outcomes 
Diabetes distress, quality of life, self-
management behaviors, SMA participation 
Criterion 1. Potential to fill 
gaps in evidence 
Criterion 2. Adoption into Clinical 
Practice and Improved Care 
Criterion 4. Patient-Centeredness 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
3 
Our multi-stakeholder BCT group endorsed a model with diabetes “veterans” co-facilitating group visits with a 
variety of diabetes care professionals, and in which patients select topics from a comprehensive curriculum according to 
their needs, preferences, and readiness for new information. The group visit curriculum should include topics on 
physical activity, healthy eating, taking medication, acceptance and coping, and social support. For this project, we 
selected the Targeted Training in Illness Management [TTIM] curriculum because it has all of these components, and 
has demonstrated effectiveness for both clinically significant reduction in HbA1c and clinically relevant improvement 
in patient-reported psychological symptoms. 41,42  The practice stakeholders want to do enhanced group visit models 
endorsed by patients. However, in their experience, SMA participation can be low, and the patient-driven SMA model 
could be burdensome to deliver and may require more resources. Practices need to know the extent to which SMA 
participation and outcomes are improved compared to the standardized approaches they tend to use now.  Patients 
need to know if it’s worthwhile to spend the time and effort participating in SMAs. Thus, BCT informed the research 
questions, patient-centered outcomes, implementation approaches, and study design. 
 
C. Study Design or Approach  
C. 1 Specific Aims:  
The purpose of this proposal is to compare diabetes SMAs led by qualified health educators using a standardized 
curriculum (standardized meaning “same for all SMA cohorts”) to a patient-driven, team-based SMA model (patient-
driven meaning “varies depending on SMA cohort needs and preferences”), in which patients involved in a given group 
(i.e., an SMA cohort) drive choice of visit topics from a comprehensive curriculum including diabetes and mental health 
topics.  The specific aims are: 
Aim 1. To implement standardized and patient-driven diabetes SMAs in multiple diverse primary care practice 
settings and refine protocols for a comparative effectiveness study. 
Aim 1a. Engage patients, caregivers, health centers, and health plans to refine the research protocol, design practice 
workflows, and plan data collection procedures and reporting to stakeholders. 
Aim 1b. Train multidisciplinary care teams in an evidence-based modular chronic disease management curriculum 
(i.e., the Targeted Training in Illness Management Program) 
Aim 1c. Implement patient-driven and standardized diabetes SMAs in 20 primary care practices (urban and rural, 
commercial and public payer, and primary care)  
Aim 2. To conduct a pragmatic, cluster randomized comparative effectiveness study of standardized versus patient-
driven diabetes SMAs in 20 diverse primary care clinics for approximately 1440 patients with diabetes 
Aim 2a. Compare changes in patient-centered outcomes, including the primary patient-centered outcome diabetes 
distress, as well as perceived autonomy support, and diabetes self-care; changes in clinical outcomes, including 
hemoglobin A1c, blood pressure, and body mass index; and out-of-pocket cost and time commitment  
Aim 2b. Compare reach and engagement of patients in patient-driven versus standardized diabetes SMAs 
Aim 2c. Evaluate practice-level outcomes, including quality of care, team-based care, and sustainability, such as 
resources required and burden to the health system   
 
C.2 Overview of approach and conceptual model 
In this pragmatic, cluster randomized comparative effectiveness trial, we will compare the reach and 
effectiveness of patient-driven diabetes SMAs to standardized diabetes SMAs for both patient-centered and clinical 
outcomes. The patient-driven SMA approach allows patients to determine visit topics and targeted care team 
involvement. The curriculum to be used is Targeted Training for Illness Management (TTIM), a 12-session modular group 
intervention adapted to be 6 longer sessions for chronic illness self-management, and has been tested in diabetes.31 
TTIM supports a positive group experience and improved chronic illness knowledge and self-management skills. The 
standardized group visit model will consist of diabetes SMAs with the full TTIM 6-session curriculum (Appendix A), led by 
a health educator (HE). The patient-driven group visit model will consist of 6 diabetes SMAs with TTIM topics selected by 
the patients; groups will be co-led by diabetes peer mentors and select members of the multidisciplinary care team 
(medical provider, behavioral  

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
4 
health provider, HE), depending on the topic. The distinguishing features of these two models are depicted in Table 2. 
The number of sessions and curriculum are identical across models; the distinction is who delivers the curriculum and 
whether patients choose session topics.  
In Aim 1, we will engage stakeholders 
to refine the study protocol and intervention 
packaging, and then provide technical 
assistance to practices to implement 
standardized or patient-driven SMAs. In Aim 2, 
we will conduct a cluster randomized 
comparative effectiveness study. There are 
three parts to this project: 1) Intervention and 
study design ; 2) Implementation; and 3) 
Research and evaluation, each of which is 
informed by the intervention conceptual model 
(Figure 1) and stakeholder engagement, 
implementation, and evaluation frameworks. 
Patient and other stakeholder engagement 
follows the analytic-deliberative model of 
stakeholder engagement. The implementation 
framework is Replicating Effective Programs 
(REP).43 The evaluation framework is RE-AIM.44  
C.2.1 Intervention conceptual model 
The conceptual model (Figure 1) 
underlying patient-driven diabetes SMAs is 
based on self-determination theory (SDT)45 and principles of whole person care.46 According to SDT, human motivation 
and behavior are a function of the social environment and the extent to which that environment supports basic 
psychological needs shown to enhance “self-determined motivation.” Considerable evidence supports SDT; studies show 
people tend to be more motivated to engage in an intervention and more likely to change their behavior when the 
intervention supports the need for autonomy (respect for choice and preference), competence (building self-efficacy, 
recognizing capacity for change), and relatedness (sense of belonging, understanding an individual’s values).47,48 SDT has 
been investigated in diabetes and key SDT factors have been found to mediate improvement in outcomes in diabetes 
self-management studies.49,50 Several elements of the patient-driven SMA model are intended to create a more 
patient-centered social and clinical environment in which the SMA curriculum is delivered, thus enhancing autonomy, 
competence, and relatedness, and increasing motivation to adopt and maintain health behavior changes needed for 
diabetes self-management.  
In the patient-driven SMA model (all elements in Figure 1), patients receive a curriculum with both diabetes 
and mental health content. In the spirit of whole-person orientation to care, this curriculum is delivered by a 
multidisciplinary care team, including health educators, behavioral health providers, medical providers, and peer 
mentors. To support autonomy, patients in a given SMA cohort select the topics they want and need at that particular 
point in time (i.e., choose the topics and the order in which they are presented). To support competence, the curriculum 
emphasizes building skills (e.g., problem-solving, goal setting, communication skills) and enhancing self-efficacy. To 
support relatedness, SMAs are co-facilitated by diabetes peer mentors, and peer mentors are available to patients 
outside of the group visit setting for individual meetings. These needs-supportive elements may enhance self-
determined motivation and help overcome known barriers to diabetes self-management behaviors; these elements are 
expected to enhance not only clinical outcomes, but also patient-centered outcomes, such as diabetes distress.  
Table 2. Features of Comparator SMA Models 
 
Standardized 
SMAs 
Patient-Driven 
SMAs 
Same for both groups 
No. of sessions 
6 (consisting of diabetes and mental health sessions) 
Educational 
components 
Diabetes and mental health with goal setting and psychosocial 
support topics (TTIM curriculum) 
SMA coordinator 
Scheduling and documentation 
Medical provider  
Medication management by billable provider (Physician 
Assistant, Nurse Practitioner, or MD/DO); 
Answer patient-specific medical questions with group present 
Distinguishing features 
Patient topic 
choice 
Order of and time spent on 
TTIM topics are set for all 
SMA cohorts 
Patients in each SMA cohort 
select order of and time spent 
on TTIM topics 
Health educator 
role 
Lead instructor for all 
educational components 
Co-instructor with peer mentor 
for non-mental health topics 
Behavioral health 
provider role 
Not involved in SMAs 
Co-instructor with peer mentor 
for mental health topics; 
Answer patient-specific 
mental/behavioral health 
questions with group present 
Peer mentor role 
Not involved in SMAs 
Co-instructor for all group 
visits; 1x1 peer access 

PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews
PCORI Cycle 3 2016 Research Plan Template                                                                       
5
In contrast, the standardized SMA model (components in Figure 1 marked by a dotted outline) includes the 
same curriculum as in the patient-driven model, but it is delivered in a standardized way (order of and time spent on 
topics are set) by health educators. Thus, distinguishing features between the patient-driven SMA model and a 
standardized model are the elements that represent a more patient-centered social and clinical environment in which 
the SMA curriculum is delivered (multidisciplinary care team with peer support and patient-driven content).
Figure 1. Patient-Driven Diabetes Shared Medical Appointment Conceptual Model
PATIENT-CENTERED SOCIAL AND CLINICAL ENVIRONMENT 
Self-Determination Needs Support
Patient-Driven Content (Autonomy)
Skills-based curriculum (Competence)
Peer Support (Relatedness)
Whole Person Orientation
Multidisciplinary Care Team
OUTCOMES
Patient-Centered 
Outcomes
Clinical Outcomes
HEALTH BEHAVIOR CHANGE
Patient self-management behaviors
DIABETES SHARED MEDICAL 
APPOINTMENT CURRICULUM
Diabetes 
Content
Mental Health 
Content
CARE TEAM PROCESS CHANGE
Relational Coordination
Chronic Illness Care Processes
C.2.2 Stakeholder Engagement, Implementation, and Evaluation Frameworks
The focus of Aim 1 is on stakeholder engagement and implementation. The focus of Aim 2 is on comparative 
effectiveness research and evaluation. The following have been shown to be useful organizing models and frameworks, 
which we will integrate in planning, delivering and evaluating our intervention:
Analytic-Deliberative Model of Stakeholder Engagement. In Aim 1, we will undertake stakeholder engagement 
and implementation planning, guided by the analytic-deliberative model (ADM) of stakeholder engagement.51 According 
to the ADM, the first step in engagement is identifying relevant stakeholders, which may include researchers, clinicians, 
healthcare providers, and patients. Stakeholders undertake a process of gathering and analyzing the evidence (the 
inputs, which include stakeholder values, experience, and literature review), deliberating (methods for combining 
evidence), and decision-making (e.g., recruitment and implementation strategies). Dr. Kwan (PI) has successfully used
engagement methods consistent with the ADM for developing and implementing study protocols.37 Thus, analysis, 
deliberating and decision making will guide agenda-setting, meeting facilitation, and refining the study protocol.
Replicating Effective Programs (REP). The implementation of SMAs will be guided by the REP framework43, used 
successfully by Dr. Waxmonsky (Co-PI) for implementation of similar programs. The use of implementation strategies 
described by REP will be used to ensure practices maintain fidelity to intervention protocols while workflows and 
procedures are adapted to the local site needs and resources. The REP implementation process involves a pre-condition 
phase (e.g., packaging intervention for training and assessment using community input), a pre-implementation phase 
(e.g., orientation, explain core elements, customize delivery, logistics planning, staff training, and technical assistance), 
an implementation phase (e.g., ongoing support and partnership, booster training, fidelity monitoring), and a 
maintenance and evolution phase (e.g., understanding requirements for sustainability). Each of these REP phases and 
elements is reflected in the SMA implementation process for this project.
RE-AIM. The evaluation will be guided by the RE-AIM framework, which is an acronym for Reach, Effectiveness, 
Adoption, Implementation, and Maintenance.44  The model grew out of the need for improved reporting on key issues 
related to implementation and external validity of health research.52 RE-AIM was developed partially as a response to 
trends toward research conducted under optimal efficacy conditions instead of in real-world, complex settings53 and has 
been applied to a wide range of conditions and study settings.54 Dr. Glasgow (Co-I), a key RE-AIM developer, will 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
6 
collaborate on its utilization and integration with the above models. Aim 2a of this project focuses on the RE-AIM 
dimension of Effectiveness (clinical and patient-centered outcomes), Aim 2b focuses on Reach (patient participation in 
SMAs), and Aim 2c focuses on Maintenance at the practice-level (sustainability factors such as resources required and 
practice burden). Adoption and implementation will be studied in future research.   
 
C.3 Research Questions and Hypotheses 
Hypotheses (Quantitative): 
Hypothesis I (Aim 2a, patient-centered outcomes). Compared to standardized diabetes SMAs, patients participating in 
patient-driven diabetes SMAs will report greater improvements in patient-centered outcomes, including diabetes 
distress (primary outcome), autonomy support, and diabetes self-management behaviors (secondary). 
Hypothesis II (Aim 2a, clinical outcomes). Compared to standardized diabetes SMAs, patients participating in patient-
driven diabetes SMAs will exhibit greater improvements in HbA1c, blood pressure, and body mass index. 
Hypothesis III (Aim 2b, patient reach and engagement). Among eligible patients offered participation in SMAs, those 
offered the patient-driven model will be more likely to accept services and will attend more scheduled sessions than 
those offered the standardized model. 
Hypothesis IV (Aim 2c, practice outcomes). Compared to standardized diabetes SMAs, practices using patient-driven 
diabetes SMAs will exhibit greater improvements in quality and team-based care. 
Qualitative Research Questions: 
QRQ-1 (Aim 2a, patient experience/autonomy support). How do patients involved in standardized and patient-driven 
diabetes SMAs describe their experience of SMA features (diabetes and mental health content, whole person 
orientation, care team member involvement)? Compared to patients in standardized SMAs, how does the experience of 
patients in patient-driven SMAs reflect themes consistent with SDT and a whole person orientation? 
QRQ-2 (Aim 2a, patient self-management). Compared to standardized diabetes SMAs, what kinds of changes do 
patients involved in patient-driven diabetes SMAs describe in diabetes self-management behaviors?  
QRQ-3 (Aim 2b, patient engagement in SMAs/costs). Compared to standardized diabetes SMAs, what reasons do 
patients involved in patient-driven diabetes SMAs cite for their level of participation? What diabetes care-related out of 
pocket and opportunity costs do patients incur under each model? 
QRQ-4 (Aim 2c, practice sustainability). What are practices’ experiences with delivering standardized and patient-driven 
SMAs? How do the models compare in terms of practice perception of resources required, burden and complexity, and 
potential for widespread uptake? 
QRQ-5 (Aim 2c, practice value). What is the overall practice perceived value for the patient-driven versus standardized 
SMAs in terms of patient participation and effects on health outcomes? For which types of patients and practices is each 
model perceived as valuable? 
 
C.4 Research Design and Methods (Criterion 3: Scientific Merit) 
 
C.4.1 Study Design. This is a pragmatic, cluster randomized comparative effectiveness trial.  Given the complexity of 
systems and processes needed to implement SMAs, and the possibility of contamination effects, randomization will be 
clustered at the practice level using covariate constrained randomization.55-57  Twenty-two practices will be randomly 
assigned to either standardized or patient-driven diabetes SMAs (10 per condition). During the 24-month 
implementation period, each practice will conduct SMAs with around 8 cohorts of 8 to 10 patients each (ultimately 
yielding up to 72 patients per practice). The initial plan is that cohorts of adult patients with T2DM will progress through 
6 weekly SMA sessions, and a new cohort will begin every 3 months, but practices can choose their own timeline. A 
mixed-methods evaluation will include quantitative (practice and patient-level surveys, electronic health records (EHR), 
and participation and intervention fidelity tracking data) and qualitative (practice and patient interviews, observation) 
components.  
Sample Size and Power Calculations: Previous power calculations used 1,440 total patients, which is not feasible given 
recruitment challenges by practices. We are expecting N (total) = 22 clinics, 1,000 patients (11 clinics and 500 patients 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
7 
per condition), allowing for ~15-20% attrition (leaving 400 patients per arm). However, we are estimating power based 
on 20 practices and provide a possible range of potential power to detect main effects. We estimated the minimum 
effect sizes detectable for different power calculations for various numbers of practices and patient sample sizes and 
intraclass correlations (ICC; see table below), with effect sizes of approximately .29 to .35 for intent-to-treat analyses of 
primary outcomes with a type-1 error rate of .05. Power for planned subgroup analyses for patient characteristics (e.g., 
mental illness, health literacy) is also shown. An effective sample size of 67 per group is required to detect a medium 
linear trend effect between groups (increasing from 0 to .5SD) using general linear mixed models with random slopes 
and intercepts.58 
Practices per arm: N 
Patients per practice: N 
With 15-20% attrition: N  
ICC 
Effective sample size 
Detectable difference (effect size)  
Power 
10 
53.4 (534 per arm) 
44.5 (445 per arm) 
3% 
193 
.29 
>80% 
10 
53.4 (534 per arm) 
44.5 (445 per arm) 
5% 
140 
.34 
>80% 
10 
46.2 (462 per arm) 
38.5 (385 per arm) 
3% 
181 
.30 
>80% 
10 
46.2 (462 per arm) 
38.25 (385 per arm) 
5% 
134 
.35 
>80% 
For subgroup analyses (for patient characteristics) 
10 
11.5 (115 per arm) 
9.6 (96 per arm) 
3% 
76 
.46 
>80% 
10 
11.5 (115 per arm) 
9.6 (96 per arm) 
5% 
67 
.49 
>80% 
 
Hypothesized Effect Size for Comparative Effectiveness for Main Patient-Centered Outcome: The hypothesized effect 
size for comparative effectiveness of patient-driven versus standardized SMAs is 0.30 (a small to medium effect), based 
on an expected difference in change in HbA1c of 0.60% between arms and a SD of 2.059 for follow-up HbA1c. Meta-
analysis shows the average effect of SMAs on HbA1c is -0.55%.6 This is the expected effect of the standardized SMAs in 
this project. A study of a similar patient-driven SMA model  to that proposed for this project led to an average change in 
HbA1c of -1.15%.30 This is the expected effect of patient-driven SMAs in this project. For instance, assuming an average 
baseline HbA1c of 9% (SD 1.2), the patient-driven model is expected to yield a follow-up HbA1c of 7.85%, and the 
standardized model is expected to yield a follow-up HbA1c of 8.45%. The increased effect of patient-driven SMAs could 
be attributed to greater patient engagement in treatment and better understanding and uptake of diabetes self-care 
behaviors, in accordance with the intervention conceptual model. Thus, a 0.60% difference in change in HbA1c between 
standardized and patient-driven SMAs is supported by both theory and evidence. Furthermore, an effect size smaller 
than 0.30 would not be clinically significant, and any increased practice burden of patient-driven SMAs would likely be 
unjustifiable. Diabetes distress is the main patient-centered outcome, but data are not directly available for distress, so 
we extrapolate from evidence on HbA1c. Evidence shows that a 0.5 SD change in HbA1c corresponds to a 1-unit change 
on the diabetes distress measure we intend to use.60,61 A 0.60% difference in change in HbA1c would correspond to a 
1.20 unit difference in DDS-17 change scores between arms. The patient-driven model may have even stronger effects 
on distress than on HbA1c. Allowing patients to choose session topics could help them focus on the areas where they 
need the most help, such as fostering family support, decreasing the sense of being overwhelmed by diabetes. 
C.4.2 Setting. This project will be centrally coordinated at the University of Colorado School of Medicine (CU-SOM) in the 
Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS), a multi-disciplinary 
research program with expertise in comparative effectiveness research, biostatistics, community engagement, and 
qualitative methods.  ACCORDS is the coordinating center for the State Networks of Colorado Ambulatory Practices and 
Partners (SNOCAP), an umbrella of primary care practice-based research networks (PBRNs), which include FQHC and 
primary care private practice members. This study will be conducted in these PBRN practices in the context of real-world 
care (see Letters of Support from PBRN practices and directors and CMHCs). The 22 participating primary care practices 
will include FQHCs and private practices. Eligible practices will have a current panel of at least 150 adult patients with 
T2DM, and access to health educators, BHPs, and diabetes peer mentors. We have established relationships and 
substantial collaborative research experience with these practices. 
 
C.4.3 Study Protocol. 
C.4.3.1 Aim 1: Refine the study protocol and implement diabetes SMAs 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
8 
C.4.3.1a Aim 1a. Engage patients, caregivers, health centers, and health plans to refine the research protocol, design 
practice workflows, and plan data collection procedures and reporting to stakeholders. 
Stakeholder Engagement Process.  Identify relevant stakeholders. Our prior efforts established strong working 
relationships with diabetes patient and provider stakeholder groups, and these existing patient and provider stakeholder 
groups will be expanded to include patient, provider and leadership representatives from all participating sites and from 
local health plans. Each participating organization will identify stakeholder representatives, including 1-2 providers and 
staff members and 1-2 patients. Each organization will invite these providers, patients, and other identified key decision 
makers to participate in stakeholder meetings for this project. Selection criteria, scripts and paper invitations will be 
provided to facilitate this process.  Stakeholder Meetings. Drs. Kwan (PI) and Waxmonsky (Co-PI) will co-facilitate 
meetings with patient, provider, and organizational leadership stakeholder representatives at least monthly during the 
first 8 months of the study, and at least quarterly for the remaining study period. These meetings are described in more 
detail under the engagement plan section below. 
 Stakeholder Engagement Objectives.  Stakeholders will be engaged in all aspects of the research from the date of 
the notice of the award. The engagement objectives are to: 
1) Refine the study protocol (RQ-2) by vetting the hypotheses, intervention packaging, study design, patient centered 
and clinical outcomes chosen by our previous BCT group, and data collection procedures with the full stakeholder group. 
See also “Engagement Plan” below. The intervention will be packaged in the form of a manual and implementation 
worksheet to instruct practices on intervention delivery and tracking (Appendix B). The manual and worksheet will guide 
clinical processes needed to identify and recruit eligible patients, set up and deliver SMAs, conduct patient assessments, 
and identify staff responsible for tasks.   
2) Plan logistics of delivering SMAs.  To optimize fidelity and consistency across practices, we will employ a practice 
coach trained in practice facilitation strategies62 to explain core elements of the intervention and customize delivery 
according to each site’s unique needs and resources. This includes designing workflows, identifying care team 
members, and establishing roles and responsibilities for SMAs. Participating practices have experience with SMAs, but 
the approach varies across practices; additionally, TTIM is a new curriculum for all practices. Every clinic is different, and 
workflows must be customized to the unique clinic context. The practice coach will work with each practice to complete 
the implementation worksheet, identify and address barriers to implementation, plan workflows and data collection 
procedures, and help identify resources needed to implement SMAs with fidelity and to execute the study protocol. This 
will help ensure this research has high internal validity, and any conclusions about comparative effectiveness accurately 
reflect the difference between standardized and patient-driven SMAs.  
The practice coach will help each practice identify SMA care team members: health educators (HEs; a general 
term referring to those qualified and selected to lead the groups, such as nurse educators or certified diabetes 
educators), prescribing providers (e.g., physicians, advanced nurse practitioners), behavioral health providers (BHPs; 
clinical psychologists, licensed clinical social workers), and peer mentors (lay health workers). Preference will be given to 
peer mentors with lived experience with diabetes, who have been patients in the respective clinics, and who have 
behavioral health experience.63,64 Peer mentors will receive ongoing training and support from study and practice staff; 
as lay health workers, supervision is necessary.65 
3) Ensure adequate SMA reimbursement mechanisms are in place.66,67 Health care financing continues to evolve, and 
thus the context in which elements of patient-driven SMAs are paid for is expected to change in coming years. For 
instance, how peer support and behavioral health services are funded today (e.g., behavioral health carve-outs, 
capitation under Medicaid) may be different in a year or two (e.g., bundled payments from accountable care 
collaboratives). The stakeholder group will review health care reimbursement strategies for the patient-driven SMAs, 
and ensure this project keeps abreast of these changes. Stakeholder representation from health plan members of the 
Colorado Prevention Alliance will facilitate understanding of local health plan-specific reimbursement mechanisms, 
which are known to vary by payer type and company. 
C.4.3.1b Aim 1b. Train multidisciplinary care teams in the TTIM curriculum 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
9 
Patient-Driven SMA Training. TTIM Training will follow a process applied successfully by Dr. Sajatovic in past work with 
complex populations with diabetes.31,41 In one-day intensives, investigators will train those who will serve as TTIM 
instructors in the patient-driven model (HEs, peer mentors, and BHPs), who will participate only in training relevant to 
their respective roles. The TTIM trainings will be led by Dr. Sajatovic, with the practice coach, Dr. Kwan and Dr. 
Waxmonsky assisting. Training will have a number of sequential components that are all designed to lead to each care 
team member’s comfort with their respective roles in delivering the curriculum (Table 3). First, the study team 
members, HEs, BHPs, and peer mentors will be provided with a semi-structured “introduction” session to get to know 
each other. Dr. Sajatovic will provide an overview of the TTIM curriculum and how TTIM fits within the SMA model. The 
SMA care team will receive a binder including all the TTIM materials to keep and review as convenient to him/her, 
including the existing detailed manual for conducting TTIM sessions (see excerpt in Appendix D).  In addition to going 
over each TTIM session, the training will include mental health and diabetes self-care topics, communication skills, group 
leading/co-leading, assistance with help-seeking pathways, and crisis management. This will involve role-play and 
repetition as needed depending on the materials and peer mentor comfort/knowledge.  Finally, the training will 
emphasize creating a patient-centered social and clinical environment that supports patient autonomy (eliciting 
preferences), competence (practicing skills) and relatedness (expressing concern for the individual).   
Standardized Curriculum SMA training.  For the standardized curriculum SMA, the Investigators will train HEs from 
standardized SMA practices in one-day intensives, similar to but separate from the patient-driven SMA practices. The 
HEs will be trained to deliver the full curriculum as the sole instructor. In addition, in both conditions, all potential 
prescribing providers who may provide medical management in SMAs will participate in a 1-hour “Lunch and Learn” 
to learn about the project and discuss their role in diabetes SMAs. 
C.4.3.1c Aim 1c. Implement patient-driven and standardized diabetes SMAs in 20 primary care practices  
In Aim 1c, 20 practices will be randomly assigned to either patient-driven or standardized SMA using covariate 
constrained randomization and then will implement SMAs, supported by technical assistance from the practice coach. 
Implementation will be overseen by Dr. Waxmonsky and managed by the implementation project manager. 
Assessment of Practice Characteristics for Covariate Constrained Randomization. Prior to randomization to condition, a 
representative for each participating practice will complete a brief assessment to collect information that will be used in 
the covariate constrained randomization (CCR) procedure described below. This information will include patient and 
practice characteristics that are likely to influence the practice’s ability to implement the intervention(s) or be associated 
with the outcome, such as practice size, practice type (FQHC, private practice), and patient panel characteristics, 
including race/ethnicity and payer mix (% Medicaid).  
Randomization to Condition.  As shown in prior work by Dr. Dickinson (Co-I), covariate constrained randomization will 
be used to enhance internal validity and achieve balanced study arms.55,68,69 The list of practice elements to be collected 
will be finalized with stakeholders as described above.  For randomization, all possible combinations of eligible practices 
into two groups will be generated using the SAS IML procedure.70   For each randomization, a balance criterion, defined 
Table 3. SMA care team member roles and responsibilities for standardized and patient-driven SMA models 
 
Care team member 
SMA Role 
Responsibilities 
Standard 
care team 
members 
for both 
SMA models 
Health Educator (RN, RD, or CDE) 
Group DSME/SMS instructor  
Deliver TTIM self-management education  
Medical Provider (MD, NP, or PA) 
Medical management 
Prescribing provider attends during the last 20 minutes of 
each group to permit reimbursement. 
SMA Coordinator 
Coordination and scheduling 
of TTIM sessions, care team 
members, and patients 
Pre- and post-visit data collection by telephone 
Schedule patients for group visits 
Coordinate sessions and schedule BHP involvement 
(Patient-driven SMAs only) 
Additional 
care team 
members 
for patient-
driven SMAs 
Behavioral Health Provider 
Instruction and Q&A for 
mental health SMA topics 
Co-lead instruction on mental/behavioral health topics 
and open-ended discussion/question and answer 
Diabetes Peer Mentor (CHW, lay 
person, patient navigator) 
Co-instructor  
Peer mentoring 
 
Co-lead instruction with HE/BHP (topic based) 
Share personal experience in group context 
1x1 support outside group visit 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
10 
as the sum of squared differences on standardized variables between arms will be computed. After examining the 
distribution of the balance criterion, an optimal set of randomizations will be identified (best 5% to 10%). From this set, 
one will be chosen using a random number generator, and the practices will be assigned to standardized or patient-
driven SMA. We have used these approaches successfully in other studies to achieve well-balanced study arms.57  
Technical assistance: The practice coach will provide technical assistance to practices as they work to execute plans 
made for delivering their assigned SMA models. The coach will also monitor progress and ensure accountability to 
timelines through regular email communication and monthly meetings with the practice team. Each participating 
practice will identify an SMA coordinator (either selected from existing staff or hired for this project) to perform 
coordination tasks necessary for ensuring fidelity to the research protocol and to ensure practices dedicate sufficient 
resources and attention to the project. The practice coach has the role of external facilitator, serving as an expert in 
practice transformation and mentoring the SMA coordinator, who serves as an internal facilitator for local 
implementation. The SMA coordinator will actively guide all daily aspects of the SMA implementation, and keep site 
leadership is informed of progress or problems in implementation.71 It is necessary for the SMA coordinator to be an 
employee of the health care organization in order to recruit patients to participate in interviews, to track participation in 
SMAs, to coordinate initial and booster training (e.g., upon staff turnover), and to have access to identified data prior to 
de-identification and transfer to the research team.  
Pilot cohort. At each practice, an initial cohort of patients will be identified and offered either the patient driven SMA or 
standardized curriculum SMA (based on site randomization). For this initial cohort, the practice coach will meet with the 
clinical team delivering the intervention or comparator intervention to assist with any potential barriers to delivery, 
clinical questions, and to garner feedback on the intervention. After each practice has had the initial cohort receive the 
intervention, any final revisions to the study protocol will be made prior to full “go-live”.   
Note on mode of delivery: Following stay at home orders put in place by state and local governments in response to the 
COVID-19 pandemic in March 2020, our participating primary care sites halted non-emergency in-person visits. The rate 
of return to in-person visits has varied for our practices based on their location, local levels of risk, and levels of risk 
deemed appropriate by the clinics themselves. As a pragmatic trial, we consider mode of delivery to be a feature driven 
primarily by practice decisions about how they will deliver care to their patient population, given their unique needs, 
preferences, capacity and local context. (Note: This perspective relates to the setting and delivery domains of the 
PRECIS-2 pragmatic trial planning framework.) As such, delivery of group visits via telehealth, while not a protocol 
change per se, is specifically allowed as long as practices maintain fidelity to core components (features required to be 
the same vs different between conditions). Importantly, mode of delivery is not a core component. Practices condicting 
SMAs via telehealth will document delivery option of groups, report specific issues seen with the telehealth delivery 
mode. Mode of delivery will be accounted for analytically, consistent with the intent-to-treat analytic framework. 
 
C.4.4.2 Aim 2: Conduct a Cluster Randomized Comparative Effectiveness Trial 
C.4.4.2a Recruitment and Retention of Study Participants (PC-2). 
As shown above in Section C.4.1, the sample size needed to detect the hypothesized effect size is 20 practices with 60 
patients each. Assuming 20% loss to follow-up among participants, each practice will be asked to enroll at least 72 
patients each. Enrollment and data collection will continue until achieving complete data for at least 60 
patients/practice; practices may enroll more than the minimum.  Recruitment strategies will be determined as part of 
Aim 1. Strategies may include identifying patients in existing diabetes registries, systematic screening and referral for 
new diagnosis of T2DM, and/or through provider referral. While strategies may vary somewhat across practices, the 
objective is to enroll as large and representative sample of eligible patients as possible in that practice. Patients will be 
enrolled in SMAs as part of their regular health care; at the practice level, this is considered quality improvement (QI) and 
similar projects have been approved as such by our Institutional Review Board. All survey data collected from patients 
will be collected as part of the SMA process, as described below, and will be made available to inform clinical decision-
making, such as which SMA topics to prioritize and to provide feedback to patients on their progress; use for research 
purposes will thus be considered secondary analysis. In our experience, making survey data relevant to clinical care is 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
11 
the best way to ensure the data are collected systematically.37 A subset of SMA patients will be recruited to participate 
in qualitative interviews and patient out of pocket cost and time commitment surveys for explanatory purposes. A 
description of this sampling plan is described in Section C.4.4.2c, Data Collection (Qualitative).  Protocols to ensure data 
are collected thoroughly and systematically will be at the practice level, managed on a daily basis by the local site SMA 
coordinator and jointly overseen by the implementation and research project managers. Practices engage patients in 
treatment as they do in real-world care (i.e., not for research purposes), using reminder and follow-up calls to encourage 
patient attendance at visits, so that the findings of this research will be directly applicable to other practice settings. 
Provider and staff study participants. Key organizational leadership, providers and other clinic staff will be invited to 
participate in practice surveys and interviews, and also participate in training and implementation.  
C.4.4.2b Interventions and Comparators. The comparator interventions are: 
1. Standardized diabetes SMA model, led by a HE using standardized curriculum content   
2. Patient-driven diabetes SMA model, co-led by a multidisciplinary care team (HE, peer mentor, and BHP) using patient-
driven curriculum content 
Selection of interventions and comparators (RQ-5). As described above, patients and other stakeholders engaged in this 
research selected the SMA features to be tested. These features characterize the patient-driven SMA model, which will 
be compared to the standardized SMA model.  The comparison condition (standardized SMA model) was selected as it 
reflects the more typical approach to SMAs in current clinical practice (a set curriculum delivered by a qualified health 
educator), according to practices engaged in this research. The selection of the TTIM curriculum – and the collaboration 
with Dr. Sajatovic, who developed and tested TTIM – was based on the topics of interest to patients engaged in BCT.  It is 
important to note that regardless of which SMA model a patient receives, patients will have access to all available care 
team members as is standard in their practice; we will assess contact with care team members outside the context of 
group visits and include it as a possible confounder in analysis.  
Description of interventions. Both SMA models will consist of 6  in-person 120 minute sessions with TTIM curriculum 
delivered in a group format with 8 to 10 patients per cohort. Patients will be asked to follow rules of discussion and sign 
confidentiality agreements at the first session. The care team members, roles, and responsibilities for both standardized 
and patient-driven SMAs are described in Table 4. Self-management education components for both models address 
diabetes management challenges as well as mental health comorbidity, particularly depression, as well as problem 
identification, and personal goal setting (Appendix A). TTIM sessions are manualized and include hand-outs that 
reinforce the oral materials, tailored to patient health literacy. In both models, patients will be recruited as determined 
in Aim 1. For instance, the SMA coordinator may invite potential participants to the SMA groups, describe the 
intervention, collect pre-visit assessment data and schedule the initial SMA session. Assessments will be conducted as 
described below. The general structure for both SMAs will be:  
1. Structured instruction/education on TTIM topics  
2. Open discussion and Q&A with SMA instructor(s)  
3. Medical management and Q&A with the medical provider (during patient-pull outs or before and after TTIM 
sessions)  
4. Wrap-up and planning for next session  
 
Standardized SMAs. Standardized diabetes SMAs consist of the full TTIM 6-session curriculum, led by HEs, with topics 
presented in the order listed in Appendix A for all cohorts. The TTIM curriculum used in this study will be slightly 
adapted to de-emphasize more acute mental health symptoms (such as psychosis) which was provided in the original 
TTIM program, and slightly increase focus on symptoms of depression and distress as prevalent impediments to diabetes 
self-management. The SMA coordinator supports patients in making and keeping medical appointments. Medical 
providers will be present to discuss patient-specific medical questions via one-on-one sessions, either during direct 
patient pull out or before or after the TTIM sessions, depending on what works best for the practice.  
Patient-driven SMAs. For the patient-driven SMAs, the curriculum will be delivered collaboratively by the 
multidisciplinary care team consisting of several professional providers (HE, medical provider, and BHP) and a lay worker 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
12 
(diabetes peer mentor). Each professional care team member is present only for the portions of the visit(s) in which they 
have a role; thus, HEs, BHPs, and medical providers will not overlap. As in the standardized model, the HE serves as the 
primary instructor, and the medical provider does medical management. Distinguishing features of patient-driven SMAs 
are the BHP, who serves as the co-instructor with the peer mentor for session topics pertinent to mental health, and the 
diabetes peer mentor, who serves as co-instructor for all visits and reinforces the curriculum by providing a personal 
experience perspective. The classic TTIM intervention includes “peer educators”, so that peer mentor roles are already 
described in the TTIM manual. BHPs help patients learn skills for coping and understanding the relationship between 
stress and health, explore the emotional side of diabetes, address adherence and motivation, and help patients engage 
in the ambivalence they feel. BHPs will deliver manualized content on TTIM content relevant to the topic of interest and 
be available for open discussion, an advantage being they can answer any questions specific to their area of expertise 
better than HEs, who do not have licensure as mental health professionals. The peer mentor will foster engagement, 
self-efficacy and behavioral modeling, address healthy behaviors, provide social and emotional support, suggest 
resources, and address questions that may arise from a “consumer” perspective. Peer mentors will be available to 
provide one-on-one support either in person or by telephone.  
Another key feature of the patient-driven model is that patients choose the topics and the order of sessions, from 
the comprehensive TTIM curriculum (Appendix A). The topics are set for the first session (introductions, setting the 
stage) and last session (debriefing and maintenance), but are otherwise open for patient prioritization. At the first 
session, patients in each SMA cohort will be guided by the co-instructors to collectively choose the order of the 
remaining TTIM sessions, and emphasis and time spent on the topics from those TTIM sessions. Depending on the topics 
selected and the order of sessions, the SMA coordinator will schedule the BHP for mental health sessions. 
C.4.4.2c Outcomes and Measures. 
We will use quantitative and qualitative methods to evaluate research questions and test hypotheses for patient and 
practice-level outcomes. Table 4 summarizes outcomes, measures, data sources, and timing and administration.  
Patient-level outcomes measures:  
Patient-reported outcomes (PC-3). Patient stakeholders selected diabetes distress as the primary patient-centered 
outcome (RQ-6). Diabetes distress will be measured using the 17-item Diabetes Distress Scale (DDS-17), a validated self-
report survey.60,61 Respondents indicate on a scale from 1 to 6 the extent to which they experience bothersome distress 
in four domains: emotional, regimen, interpersonal, and healthcare navigation burden. The DDS-17 has been 
demonstrated to be strongly related to and prospectively predictive of diabetes self-management behaviors as well as 
glycemic control, and has discriminant validity for depression measures.14 Additional patient-reported outcomes 
measures to be administered were selected by patients and/or reflect SDT constructs from the conceptual model; all are 
established valid and reliable self-report measures (IR-4). Perceived autonomy support in health care settings and self-
efficacy (SDT constructs) will be measured using the 6-item Health Care Climate Questionnaire (HCCQ) and the 4 item 
Perceived Competence for Diabetes Scale (PCDS).73 Diabetes self-management behaviors will be measured using the 
Summary of Diabetes Self-Care Activities (SDSCA). The 11-item SDSCA assesses self-reported dietary adherence, physical 
activity, and medication adherence, and is the most widely used and validated brief patient report scale for diabetes 
self-management behaviors.74 Patient out of pocket cost and time commitments will be collected using questions drawn 
from existing surveys. Out of pocket cost measures will use questions drawn from the Medical Expenditure Panel Survey 
and the National Health Interview Survey.75 Measures of the amount of time patients spend receiving, waiting for, and 
traveling to receive medical services will be drawn from the American Time Use Survey.76 Health literacy, a potential 
moderator, will be measured using the validated 3-item Health Literacy Scale.    
Patient-level clinical outcomes (Aim 2a). We will gather clinical outcomes data (HbA1c, blood pressure, body mass 
index) from participating practices’ EHRs. All encounter data (dates, locations, visit and provider types, diagnosis codes), 
lab results and vital signs measures from 9 months before through 9 months after each patient’s initial SMA visit will be 
requested.  Diagnosis codes (ICD-9/10) will be used to compute a comorbidity index77 and to corroborate diagnosis of 
mental illness. Prescription data will be used to assess insulin dependence. 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
13 
Patient reach and engagement in SMAs (Aim 2b). We will assess reach and engagement in terms of acceptance (i.e., 
what proportion of patients agree to participate in SMAs?) and participation (i.e., what proportion of SMAs do patients 
attend?), defined by Raue and Sirey.78 Attendance at each session and total attendance for each of the 6 sessions, as well 
as any reasons for dropout, will be recorded by the SMA coordinators in a spreadsheet.  
 
Practice-level outcomes measures (Aim 2c).  
Practice quality of care, team-based care, and resource requirements. We will administer several validated 
practice-level measures to all practice staff and providers, as summarized in Table 5. Practice staff and providers in 
participating practices will be asked to complete survey measures assessing aspects of practice context (which often 
influence efforts to improve diabetes care79), including measures of relational coordination (using the Relational 
Coordination Survey,80 a measure of team-based care and important factor in chronic illness care in primary care 
teams81), practice CCM-consistent care (using the Assessment of Chronic Illness Care [ACIC])82, and role in SMAs. Those 
involved in SMAs will also complete measures of SMA resource requirements using a time-driven activity-based costing 
[TDABC] framework83 to assess staff time use, workflows, required materials and supplies, and other resources needed 
to deliver each SMA model, distinguishing between early and late implementation phases. To assess practice culture, the 
Practice Culture Assessment will be administered to all practice staff. 
Table 4. Outcomes, Measures and Data Sources 
Outcome 
Domain 
Construct 
Source 
Metric/Measure 
Timing  
Respondent
/ Unit 
Administered 
/Collected by 
Patient Reach 
and 
Engagement 
Service acceptance  Tracking 
spreadsheet 
Attendance at initial SMA 
session among all invited 
Initial SMA 
session 
Patient 
SMA coordinator 
Service 
engagement 
Check-in sheets 
#/% and types of sessions 
attended 
Weekly 
Patient 
SMA coordinator 
Interviews 
Patient reasons for 
participation/non-
participation 
Within 3 weeks 
of last session 
Patient 
Qualitative Co-I and 
professional 
research assistant 
(PRA) 
Patient 
demographics 
EHR 
Age, gender, insurance, 
race/ethnicity 
Collected during 
routine care  
Patient 
Practice data 
analyst 
Patient clinical 
status  
EHR  
Comorbidity index77; 
Insulin dependence, 
mental illness 
Collected during 
routine care  
Patient 
Practice data 
analyst 
Patient-Level 
Effectiveness 
Outcomes 
Diabetes distress 
Survey 
DDS-17 
1st and last 
session 
Patient 
SMA coordinator 
Autonomy 
Support, self 
efficacy  
Survey 
HCCQ, PCDC 
1st and last 
session  
Patient 
SMA coordinator 
Self-care behaviors 
Survey 
Summary of Diabetes Self-
Care Activities74 
1st and last 
session 
Patient 
SMA coordinator 
 
 
 
 
 
 
Health literacy 
(moderator) 
Survey 
Chew Health Literacy 
Scale 
1st session 
Patient 
SMA coordinator 
Clinical outcomes 
EHR 
HbA1c, blood pressure, 
body mass index 
Collected during 
routine care 
(~quarterly)  
Patient visit 
Data analyst 
Patient experience 
Qualitative 
Interviews 
Interview guide  
> 3 weeks of last 
session 
Patient 
Qualitative Co-I and 
PRA 
Practice-Level 
Fidelity, 
Effectiveness 
Intervention 
Fidelity  
Tracking 
spreadsheet, 
observation 
TTIM Fidelity tool 
Intermittent, 
ongoing 
Clinical 
team, 
patients 
Co-PI and PRA 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
14 
Table 4. Outcomes, Measures and Data Sources 
and 
Sustainability  
Team-based care 
Survey 
Relational coordination 
survey80  
Baseline, 9, 24 
months 
Clinical team 
PRA 
Quality of care 
Survey 
Assessment of Chronic 
Illness Care (ACIC)82  
Baseline, 9, 24 
months 
Clinical team 
PRA 
Practice culture 
Survey 
Practice Culture 
Assessment (PCA) 
Baseline, 9, 24 
months 
Clinical team 
PRA 
Perceived value, 
resources 
required, burden 
to practices 
Qualitative 
Interviews 
Interview guide 
Baseline, 9, 24 
months 
Clinical team 
Qualitative Co-I and 
PRA 
Survey 
TDABC framework.83 
Baseline, 9, 24 
months 
Clinical team 
Qualitative Co-I and 
PRA 
Quantitative data collection. Data collection procedures are as follows: Patient participation data will be tracked by the 
SMA coordinator in a spreadsheet. At each SMA session, the instructor will complete a check-in sheet noting the TTIM 
topics covered and any care team members and patients in attendance, and provide this information to the SMA 
coordinator for data entry. Most patient surveys will be administered to patients by the SMA coordinator (during SMA, 
by telephone or online, as determined in Aim 1a) prior to the first and last SMA sessions; As surveys are clinically 
relevant, they will be scored and made available to patients and instructors to inform discussion (and for patient-driven 
SMAs, topic selection) during the SMAs. Patient cost and time surveys will be administered separately by a PRA with the 
patients recruited for patient interviews, as these cannot justifiably be considered part of clinical care. The SMA 
coordinator will attempt to collect surveys for all patients, including those who stop attending sessions. The SMA 
coordinator will be responsible for survey administration, scoring and entry into a local database (e.g., a custom EHR 
template in the patient’s record or a separate database with the patient’s medical record number [MRN]37). This ensures 
survey data can be linked at the patient level with clinical data. MRN will be replaced with a random identifier prior to 
transfer to the study team (IR-2). 
Clinical outcomes data for each participating patient will be extracted from the participating practices’ EHR for the 
period 9 months before and after each patient’s involvement in SMAs using chart review and extracts from practice 
backend EHR databases. EHR extracts will be requested for all patients enrolled to date at the mid-point and end of the 
implementation phase. Patients with diabetes are typically seen in primary care every 3 months, or more often if poorly 
controlled, and data on HbA1c, body mass index (BMI), and blood pressure are collected routinely at these visits. Using 
data collected in the course of routine care is a pragmatic feature of this project, reducing costs and burden to practices 
and patients. Because all data will be gathered from the practice EHRs, availability of data will not be dependent on 
participation in interventions, allowing robust estimates of effectiveness of interventions among those for whom they 
are intended as well as sub-analyses among those who participate. These EHR extracts will include patient survey data, 
as described above. The SMA coordinator will provide a list of patients to a practice data analyst (an existing role in  
participating practices). The analyst will extract the requested data into flat files following specifications provided by the 
research team, strip direct identifiers to create a HIPAA limited data set, and create a random unique patient identifier. 
Data will be transferred to the research team using a secure cloud-based encrypted transfer mechanism (Egnyte.com), 
cleaned, and standardized across practices according to the OMOP common data model using middleware developed by 
Dr. Kwan and other local informatics colleagues.84 Extract specifications will be refined in collaboration with analysts 
following the initial extract and review of data quality following recommendations for data quality checking in CER (e.g., 
assessing attribute domain constraints including ranges, relational integrity rules, historical data rules including temporal 
components, and missingness).85 HIPAA data use agreements will be executed with participating practices prior to 
transfer of data. Drs. Kwan, Nease and Dickinson routinely use this approach through our data network infrastructure.  
Practice surveys will be administered at baseline (pre-implementation state), 6-9 months (early impressions) and 
within 2 months of completion of SMAs (final impressions) in the implementation phase; practices will be given the 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
15 
choice of paper or electronic data collection using RedCap. The goal will be to collect surveys from at least 70% of all 
practice staff and providers, and 100% of those involved in diabetes SMAs in the prior 6 months.  
Observation field notes and intervention fidelity measures. Fidelity to SMA processes, content, and format will be 
evaluated by non-interventionist study staff using a checklist for fidelity monitoring (Appendix C). A randomly sampled 
10% of SMA sessions will be observed and coded for fidelity to determine if sessions covered relevant TTIM topics, 
format was appropriate, and if sufficient time was devoted to the discussion and Q&A session.  
 
Qualitative data collection. We will conduct interviews with patients and practice staff and providers involved in SMAs 
to gather data for answering qualitative research questions on SMA experience, value and sustainability. 
Patient interviews. Interviews will be conducted by a qualitative PRA using a semi-structured interview guide, under 
the guidance of Dr. Holtrop. Interviews will assess patient experience of SMAs specifically and diabetes care more 
generally. Probes will include the various elements of SMAs and which were most valuable (emphasizing exploration of 
the elements thought to reflect SDT constructs), reasons for participation or non-participation in SMAs, barriers and 
facilitators to participation, experience with care team members, and effects on self-management behaviors.  
Interviews will be conducted with 3-5 patients selected from each of the 20 participating practices. To avoid 
contaminating the intervention, patients will be interviewed after completion of their SMA experience. This will mostly 
include patients who have completed the program, but also include patients who decide to discontinue their 
participation. In this way, a variety of participation experiences can be explored. Participants will be purposefully 
selected to reflect a variety of ages, race/ethnicity, genders, and participation experiences (high and low participation) 
as well as the two study arms (standard and patient-driven SMAs). For each SMA, the class roster will be reviewed by a 
professional research assistant (PRA) and the SMA coordinator. One or two patients per practice in the initial cohort of 
participants will be identified. The SMA coordinator will use an opt-out procedure approved by our IRB and successfully 
used in our prior research, mailing the participant a letter of invitation for the interview and that they can opt out of 
participation. If the patient declines, s/he will not be contacted by the PRA for interview scheduling. The PRA will 
continue to schedule and interview participants in future cohorts until at least the minimum number of patients is 
obtained (35-40 per condition). Total interview numbers are expected to vary depending on saturation of themes across 
arms. These same patients will be invited to complete the cost and time commitment survey. Participants will be 
compensated with a $50 gift card for the 60-minute interview and cost survey completion. 
Practice interviews on perceptions of value and sustainability. One-on-one, in-person key informant interviews86 
will be conducted with practice members involved in SMAs at baseline, midpoint and endpoint post-implementation. 
Three to five key informants per practice (all care team members involved in SMAs, varies by condition) at each time 
period will be interviewed individually, in person by Dr. Holtrop (Co-I) or a PRA. Interview guides will cover practice 
perceptions of the value and sustainability (burden, complexity and potential for widespread uptake) of patient-driven 
and standardized diabetes SMAs. A semi-structured interview guide will be developed for each time period. Baseline 
interviews will focus on importance and interest in the upcoming SMAs, factors thought to affect adoption of the SMAs, 
and anticipated patient response to the SMAs. Midpoint and endpoint interviews will elicit the participant’s experiences 
with the SMAs, including a cognitive task analysis of the intervention as delivered in the practice, to provide a detailed 
understanding of fidelity and any possible adaptations, while illuminating gaps in understanding.87 Endpoint interviews 
will also specifically focus on recommendations for other practices and plans for continuing SMAs. This information will 
be considered in light of contextual factors such as practice structure and patient panel characteristics.  
C.4.4.2d Analytic Plan (Methodology Standard IR-3) 
The analytic plan includes both Quantitative (section C4.4.2d.1) and Qualitative (section C4.4.2d.2) components. 
C4.4.2d.1 General Quantitative Approaches. For this cluster randomized trial, descriptive statistics will be computed for 
baseline patient and practice characteristics, initially reporting on differences between: (1) different intervention arms 
and (2) patient dropouts vs. non-dropouts (MD-4). Patient-level covariates will be screened in bivariate analyses and 
included in multivariate analysis if related to the outcome at p<.2, are associated with dropout, or used in the 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
16 
randomization procedure.88 Covariates (to adjust for potential confounding) and potential moderators will include the 
following: mode of delivery of SMA (in-person vs. telehealth), age, gender, race/ethnicity, comorbidity index, insulin 
dependence, baseline diabetes distress, health literacy, and mental illness. Recent literature on cluster randomized trials 
notes that imbalance in covariates is more of a problem than for individually randomized trials, and covariates should be 
included to improve precision and should be selected prior to analysis.55,88,89 In the event normality assumptions are not 
met, we will use transformations to normalize distributions, ordinal or Poisson regression where appropriate, and/or the 
appropriate link function (e.g. logit link for dichotomized measures).90-92 We will employ intent to treat analyses using 
general (generalized) linear mixed models (GLMMs) to incorporate data structures that are both hierarchical and 
longitudinal.91,93-97 Hypothesis tests will be two-sided with ɲ=.05 or p values reported. Goodness of fit statistics and 
model fitting diagnostics will be used to assess for influential points, outliers, overdispersion and heteroscedasticity and 
to evaluate alternative model specifications.90 All statistical analyses will be performed using SAS version 9.4 (SAS 
Institute Inc., Cary, N.C.). 
Missing Data (MD-2). Prior to beginning analyses described below to address study hypotheses, we will examine the 
data carefully to determine whether patterns of missing are ignorable (MCAR or MAR) or non-ignorable (MNAR).98-101 If 
so, we will employ likelihood–based methods that utilize all available data (MD-4), adjusting for covariates that are 
associated with missingness (MD-3). If missingness is non-ignorable we will employ pattern mixture models. 102 
Sensitivity analyses will be carried out using multiple imputation approaches (MD-5). 
Quantitative analysis plan. The following analyses will be carried out by Dr. Dickinson and the quantitative analyst. 
In recent literature on cluster randomized trials, general (or generalized) linear mixed models, adjusted for covariates 
are recommended for analysis of cluster randomized trials,89,103 even after using such procedures as constrained 
randomization.54   Likelihood based models using all available data are the preferred method for analyzing longitudinal 
data with dropout under MAR conditions.104-107 This will be our primary analysis; however, we will examine change 
scores as outcomes, adjusting for baseline, in sensitivity analyses. 
To test hypothesis I (comparing patient-centered outcomes): The outcomes for these analyses will be patients’ 
scores on each of these measures at baseline and follow-up (1st and last session). The structure of the data is hierarchical 
(patients nested within practices) and  repeated measures on patients over time. In the basic mixed effects model 
shown below, Ytij is the score for patient i at time t (baseline: time=0, or follow-up: time=1) in practice j. Intervention is 
an indicator variable: 0=standard SMA, 1=patient-driven SMA. Covariates will be included in all models but are not 
shown here for ease of understanding. 
  
  
Ytij = J000 + J010 (intervention) + J100 (time) + J110 (intervention x time) + u00j   +  r0ij  +   Htij 
 
where J000 represents the initial status for standard SMA;  J010 represents the baseline difference between the patient 
driven SMA practices and standard SMA practices (should be close to 0);  J100 is the pre-post change in scores for the 
standard SMA practices; J110 is the difference in pre-post change (i.e. difference in difference) for standard vs. patient-
driven SMA practices. rtij is a patient random effect and U00j is a practice random effect, independent of rtij and assumed 
to have a bivariate normal distribution over practices;  Htij is residual variance. Thus, the intervention effect of patient-
driven vs standard SMAs can be tested as H0: J110=0 vs H1: J110тϬ͘ 
In the model above we use practice level random effects. However, it is possible that groups (within practices) are a 
greater source of variation than practices per se. Therefore, we will estimate variance components for groups within 
practices and retain both random effects if there is sufficient variability. If it is not possible to retain both group and 
practice random effects, we will test each separately and examine model fit and variance components to determine 
which to retain.  
To test hypothesis II (comparing clinical outcomes): Outcomes for these analyses will be obtained from EHR data 
covering 9 months’ post-baseline for each patient. The baseline measure for each clinical outcome will be defined as the 
last recorded value prior to enrollment in the study. All subsequent clinical measures through 9 months after enrollment 
will be used in analysis. Analytic approaches will be similar to those described for hypothesis 1, with time will be coded 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
17 
as time since baseline. Random coefficient growth curve models will be used to determine whether trajectories for 
patients in standard SMA practices differ from patients in patient-driven SMA practices (time x arm interaction term in 
mixed model notation). 
Hypothesis III. Compared to standardized diabetes SMAs, practices using patient-driven diabetes SMAs will exhibit 
greater improvements in quality and team-based care, as measured by the Relational Coordination Survey and the 
Assessment of Chronic Illness Care. 
To test hypothesis III (comparing patient reach and engagement). The outcomes for these analyses will be 1) 
acceptance of services (Y/N), and 2) number of SMA sessions attended. General or generalized linear mixed effects 
models (depending on distribution of outcomes variable) with a practice random effect will be used for analysis. In 
addition to including covariates as described above, we will test whether mental illness is a moderator of intervention 
effectiveness. This will involve including a main effect for mental illness, along with an arm x mental illness interaction 
term (mixed model notation). 
To test hypothesis IV (comparing practice quality and team based care): Outcomes for this analysis will be scores 
on the relational coordination survey and the ACIC at baseline, 9, and 24 months. Analytic approaches will be similar to 
those described for hypotheses I and II. 
Heterogeneity of treatment effects (HTE) (HT-2). It is not known at this time whether patient-driven SMAs will 
benefit certain patients more than others. Thus, in addition to the analyses described above, we will conduct 
exploratory analyses to test for potential effect modification (moderator of intervention effectiveness) by selected 
patient characteristics (HT-1). While this study may be powered to detect effect modification with large effect sizes, the 
planned sample size of 1000 enrolled will allow only for testing effect modification of patient characteristics 
representing subgroups of an average of 10 patients per practice with that characteristic. For example, to do a subgroup 
analysis of patients with mental illness, we would need an average of 10 patients with mental illness per practice to test 
this effect. This analysis is described just above for analysis of hypothesis IV with mental illness as a potential moderator; 
mental illness comorbidity is our primary target for HTE analyses and will be examined for every hypothesis above.  
Additional sub-populations of interest are defined by gender, Hispanic ethnicity, and health literacy; existing evidence 
suggests possible differential participation and effectiveness for these groups.29,108,109 The general hypothesis for these 
moderator analyses is as follows: The effects of the patient-driven vs standardized SMAs on diabetes distress (and other 
primary and secondary outcomes) will differ for patients with mental illness compared to patients without mental illness 
(male vs. female, Hispanic ethnicity vs not, high vs. low health literacy, food security, cohort conducted in English or 
Spanish, virtual vs. in-person, insulin dependence). To assess for possible heterogeneity of treatment effects by mode of 
delivery (effect modification, moderator) additional analyses will be performed including the appropriate two and three-
way interactions between treatment arm and mode: arm x modality interaction for single outcomes, arm x time x 
modality along with relevant two-way interactions for longitudinal analyses. Additionally, sensitivity analyses will 
performed to further explore delivery mode. Additionally, we will examine variability between SMA groups separately 
for standard and patient-driven arms to determine if the group itself (especially in the patient-driven arm) is an 
important contributor to differences in outcomes. 
 For hypotheses I, II and IV, moderator analyses involve inclusion of main effect for time, arm, moderator variable, 
time x arm, time x moderator, arm x moderator, and time x arm x moderator interaction term in the model (HT-3). The 
three-way interaction term tests for differential intervention effectiveness in subgroups identified by the moderator 
variable.  Due to the exploratory nature of these analyses, we do not plan to adjust for multiple comparisons in 
moderator analyses using the subgroups defined above. However, interpretation of results will include reporting on all 
subgroup analyses and take into account the number of subgroup analyses performed (HT-4). 
Patient cost/time and practice resource/time data analysis. The following analyses will be conducted by Dr. Gritz. 
Patient cost/time and practice resource and staff time use data will be examined using simple descriptive measures, 
including minimum and maximum values (range) and means. Descriptive measures will be calculated for practices in 
each arm of the study as well as for each type of practice and by level of patient participation. To examine the extent to 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
18 
which practice resources are related to patient participation in SMAs, we will examine multiple values calculated for 
each measure, including a practice-level value, per-patient that agrees to complete these measures, and the value of the 
measure per-patient that completes different numbers of sessions. Outlier values of these measures will be confirmed 
with practices prior to the final analysis of these measures. Results from the analysis of practice resources and staff time 
use will not involve formal statistical tests of differences, as these data will be used for descriptive rather than 
comparative purposes. 
 
C4.4.2d.2 Qualitative Analysis  
To answer the qualitative research questions (QRQ 1-5, Section C.3), a qualitative analysis of practice and patient 
interview data will be conducted under the direction of Dr. Holtrop with assistance from qualitative PRAs (team of 2-3) 
and with ongoing input and direction from the core study team. Interview data will be transcribed, cleaned and entered 
in the ATLAS.ti qualitative software program. Qualitative analysis proceeds as a small group process to capitalize on the 
triangulation of the data across analysts, resulting in a rich result, accurate to the data. For all analyses, we will begin 
with a grounded hermeneutic editing approach to the data.110 This approach will help to identify themes that are 
“grounded” or developed from an interpretation of the data. The analyst team will read through 5 to 10 interviews and 
together discuss and determine the key themes and the associated definitions and labels (“codes”). These codes will be 
vetted with the study team and stakeholder representatives. After initial codes are established, the analysts will code – 
first together, then independently – the data using a coding and editing approach as outlined by Addison, and will 
compare and reconcile coding until achieving a high degree (> 80%) of conceptual inter-rater reliability. The text will be 
sectioned into segments representing categories of responses. 
To answer QRQ 1-3 (patient experience, effects on self-management, and engagement), data from the patient 
interviews and program observation field note data will be used in the qualitative analytic approach noted directly 
above, with focus on the grounded theory analysis. To analyze the patient data, codes will be created and utilized to 
interpret the patient data. Within ATLAS.ti, patient interviews will be grouped into two families: standardized SMA and 
patient-driven SMA participants. We will examine the codes across these two groups by comparison of quotation 
reports completed by group to determine if patients in the two SMA conditions express different SMA experiences and 
changes in self-management behaviors.  A priori coding using a template framework will be used in both groups to 
identify evidence of self-determination theory (SDT) constructs. A specific set of codes (i.e., competence, autonomy, 
relatedness) will be used to gather evidence to support or refute patient expression of SDT constructs interacting with 
the intervention. Finally, examination of patient expressions of reasons for participation (or non-participation) will be 
examined with data regarding actual participation to corroborate and explain quantitative results. The final phase 
consists of preparing interpretive summaries detailing the findings of prior phases. All phases of data processing and 
analysis will be cross-checked to ensure consistency in application of coding and classification procedures.  
The analysis of QRQ-4 (practice sustainability) and QRQ-5 (practice value) will proceed in two directions: 1) Develop 
overarching themes across all practices; 2) identify factors (practice context, type, intervention arm) that explain 
perceived value and sustainability of SMAs.  Using the approach described above, overarching themes related to the 
data emerge to describe the experience of the practice members as they interact with the intervention. This analysis will 
discover if there are key underlying characteristics of the providers or practices such as belief systems or mindsets, or if 
there are practical reasons that make SMAs work or not work and to what extent. Ongoing meetings will be held with 
the overall team considering existing literature and associated experiences such that corroborating/legitimating will 
occur to seek out additional data to confirm or refute insights from the analysis. After initial analysis has identified data 
to support one theme or interpretation, particular effort will be devoted to finding negative or disconfirming evidence. 
Practice members from  practices for each type (FQHC, private) will be selected for checking and revision of the thematic 
groupings prior to final coding.  
Second, a matrix approach will be used to examine specific identified factors across practice “cases” to identify if 
there is a pattern of configurations that may explain results. Each practice will be represented as a case (row in the 
matrix) and the conditions of interest (practice types and context and intervention characteristics) will form the 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
19 
columns. These configurations of conditions with overall perceived value will be compared using qualitative methods 
and the formal technique of qualitative comparative analysis.111-113 This analysis will help identify factors important to 
the relative effectiveness of SMAs across model types (standard or patient-driven) and practice characteristics. Dr. 
Holtrop has experience with this approach and has used it in previous studies.114  
 
Reporting plan (IR-6). The fidelity and participation checklists will capture differential participation, which may influence 
the effectiveness of the comparators. Progress reports will detail recruitment, enrollment, completeness of follow-up by 
intervention arm, internal consistency of measures, primary analysis of all outcomes, and any exploratory analyses. For 
manuscripts, practice and patient recruitment and retention will be tracked and reported in accordance with the 
CONSORT statement for cluster randomized trials and the RE-AIM extended CONSORT diagram.115,116   
D. Patient Population and Subgroups (RQ-3 and RQ-4) 
Up to 1000 patients (45.5 patients/practice * 22 
practices) will be enrolled in either standardized or 
patient-driven SMAs and will contribute data to the 
evaluation. Inclusion criteria for patients include adults 
at least 18 years old with Type II diabetes, and who 
receive care in participating practices. Exclusion criteria 
include pregnancy or plans to become pregnant in the 
next six months, limited cognitive capacity such as that 
due to dementia or a developmental disorder, less than 
one year life expectancy, or plans to leave the area in 
the next year. These criteria will be finalized with the 
stakeholders.  Subgroups include patients with 
comorbid mental illness (e.g., depression, anxiety, 
bipolar disorder). We anticipate each practice will need 
to invite at least 150 patients to participate in SMAs in 
order to successfully enroll 46 patients who agree to 
participate (50% treatment acceptance rate) and retain 
at least 36 patients who contribute complete data to 
the evaluation (15-20% attrition). Barriers to enrollment will be addressed as part of stakeholder meetings throughout 
the implementation period (see engagement plan). Practices will use their usual procedures to maximize patient 
attendance at visits, including calling patients to remind them of upcoming visits, arranging for transportation (often 
needed for patients in FQHCs), and calling patients who do not show for a scheduled visit to assess barriers to and 
encourage future attendance (PC-2, retention). We will also attempt to collect all follow-up survey data from patients 
who discontinue participation, through email or telephone. 
Table 5. Recruitment Plan for Prospective Studies: 
Estimated number of potentially eligible 
study participants (describe how this 
number was determined [e.g., EHR, claims 
data, clinic logs, administrative data, 
other]): 
Participating practices see on 
average ~500 unique adult 
patients/year with diabetes 
(estimated from EHR data) 
Total number of study participants 
expected to be screened: 
N/A (screening does not apply) 
Total number of study participants 
expected to be eligible of those screened: 
N/A 
Target sample size (use same number 
stated in milestones): 
1000 patients (45.5/practice *22 
practices) 
If applicable, total number of 
practices/centers that will enroll 
participants: 
22 
Projected month first participant enrolled 
(month after project initiation): 
Month 9 (pilot cohort) 
Projected month last participant enrolled 
(month after project initiation): 
Month 45  
Projected rate of enrollment (anticipated 
number enrolled per month of enrollment 
period): 
1 new cohort of 8-10 patients per 
practice every 3 months 
Estimated percentage of participant 
dropout: 
15-20% 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
20 
E. Investigators and Environment (Criterion 4) 
As noted above, this study will take place in the CU-SOM’s 
ACCORDS in collaboration with member practices of the 
SNOCAP PBRNs. The research team has considerable expertise 
and experience in practice-based research, diabetes self-
management interventions, study design and analysis, 
implementation science, stakeholder engagement, mental and 
behavioral health, and mixed methods research. Dr. Kwan, the 
PI, is a social health psychologist and health services 
researcher with experience in mixed methods research and 
application of psychological theory to the design and testing of 
interventions for chronic disease management in primary care 
settings. She is the Associate Director of SAFTINet, a 
distributed data network and SNOCAP-affiliated PBRN with 
infrastructure linking clinical, claims and patient-reported data. 
With the support of the Co-PI and Co-investigators, with whom 
she has a track record of successful collaboration, she is well-prepared to lead this study. Dr. Waxmonsky, the Co-PI, is a 
clinical psychologist and director of research and evaluation at the Jefferson Center for Mental Health with a history of 
successful collaboration with the clinics engaged in this study. She has extensive experience in stakeholder engagement, 
mental health and primary care, and leading implementation of similar chronic disease management programs.  
Dr. Glasgow is well-known for his seminal work on the RE-AIM framework, and implementation and testing of 
diabetes SMS interventions. Dr. Nease is a family medicine physician with clinical experience with diabetes, community 
engagement core leader for ACCORDS and Colorado’s clinical and translation science award (CTSA), and the director of 
SNOCAP.  Dr. Dickinson is a senior biostatistician with experience in cluster randomized trials and multilevel modeling. 
Dr. Holtrop is an experienced mixed-methods researcher with expertise in qualitative methods, and has published 
widely on interventions based on the CCM in primary care. Dr. Gritz is an economist with experience in collecting and 
analyzing patient out of pocket costs and time use, as well as practice resource and staff time use data for diabetes 
prevention and other interventions. Dr. Sajatovic, psychiatrist and health services researcher, led the TTIM development 
and implementation, including intervention packaging, training, and fidelity monitoring.  Dr. Ritchie is a clinical health 
psychologist with experience coordinating and leading diabetes SMAs at participating practices.  
Selection of study sites. The participating organizations have longstanding relationships around collaboration in 
research and QI as part of our PBRNs, and for medical and behavioral healthcare integration initiatives in Colorado. They 
expressed considerable interest in participating as they see the patient-driven SMAs as an opportunity to improve 
diabetes care in their practices. Furthermore, they perceive both SMA models as readily fitting into existing clinical 
workflows and diabetes care initiatives, and intend to disseminate patient-driven SMAs to other practices if successful 
(see Letters of Support). Integrated behavioral health and primary care is in place in all practices that have signed letters 
of support. Peer mentor resources are available through contracts with Community Mental Health Clinics for practices 
that do not already have these resources on staff. Therefore, availability of personnel is not expected to be a barrier to 
participation.  
 
F.  Engagement Plan (Criterion 6)  
This proposal emerged from strong patient and practice stakeholder engagement using BCT. Three 
patients/caregivers and three providers participated in preparing this proposal, and intend to continue upon funding.  
Our BCT group advocated for studying effective and engaging models of diabetes group visits, an issue of importance to 
many of our PBRN practices.  Our patient and practice stakeholders indicated group visits can be plagued by low 
attendance (which makes practices reluctant to consistently offer SMAs) and “information overload” (which makes 
patients reluctant to participate and may exacerbate diabetes distress).39 Still, patients felt SMAs were an appropriate 
Table 6. Estimated Final Racial/Ethnic and Gender Enrollment Table 
Race 
Male (N) 
Female (N) 
Total (N) 
American Indian/Alaska 
Native 
4 
4 
8 
Asian 
15 
15 
30 
Black/African American 
27 
27 
54 
Hawaiian/Pacific Islander 
2 
2 
4 
White 
444 
444 
888 
Multi-race 
8 
8 
16 
Total 
500 
500 
1.000 
Ethnicity 
Male (N) 
Female (N) 
Total (N) 
Hispanic (Latino/Latina) 
290 
290 
580 
Non-Hispanic 
210 
210 
420 
Total 
500 
500 
1,000 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
21 
forum for DSME/SMS, and practices appreciate the efficiency of offering DSME/SMS in a group setting. The analytic-
deliberative model of stakeholder engagement (described above) provides a backdrop for ensuring the six engagement 
principles continue to be front and center in our stakeholder engagement efforts. Decision making will occur at multiple 
levels: high-level decisions (e.g., the overall targeted patient population) will be made at stakeholder meetings, while 
local decisions (e.g., specific workflows to identify the targeted population) will be made at local implementation 
meetings and in consultation with participating providers and staff. Patients and other stakeholders will participate at all 
levels of decision making, providing input on planning, implementation, and dissemination of findings. For all phases of 
the project, the engagement process will involve 1) collaborative agenda setting for stakeholder meetings; 2) meetings 
led by skilled facilitators who actively invite all participants to share their expertise, experience and perspectives; 3) 
circulating meeting summaries to all stakeholders with major decisions made and remaining questions; and 4) inviting 
feedback on decisions through email or individual meetings (for those less comfortable in group discussions). This helps 
ensure transparency, trust, and honesty; eliciting perspectives of all partners contributes to co-learning. The research 
team will ensure agendas are clearly defined and well-organized, and held in convenient, accessible locations; this 
respects the stakeholders’ time.  Communication outside group settings will be done at a time convenient to partners, 
using their preferred communication strategy. Patient and practice stakeholder partners will receive compensation for 
participation in stakeholder and implementation planning meetings, reviewing materials and providing input on 
decisions as described in the budget justification.  
1. PLANNING THE STUDY: Stakeholder engagement in planning to date has been described above. The stakeholders will 
inform the process of implementation of patient-driven and standardized SMAs, help refine the research protocol 
(including outcomes assessment, evaluating fidelity to the SMA models, and recruitment), and contribute to 
interpretation of findings and dissemination in the literature and in practice. Details about stakeholder engagement in 
planning the implementation has been described in Aim 1a.  
2. CONDUCTING THE STUDY: Stakeholder partners (patients and practice representatives) will participate in the study 
conduct by reviewing enrollment numbers and data collection, and suggesting changes in approach should the rate of 
enrollment lag behind milestone goals - each providing their own perspective. There are often unforeseen barriers and 
challenges in implementation; stakeholders can also contribute to troubleshooting these barriers. The stakeholders may 
have to decide, for instance, the extent to which elements determine to be required elements need to be adapted or 
modified. Health plan stakeholders will inform reimbursement plans. Quarterly stakeholder conference calls are planned 
throughout the 24-month implementation phase.   
3. DISSEMINATING THE STUDY RESULTS: Patient partners will be engaged in dissemination by 1) contributing to 
interpretation of results from the patient and family perspective, 2) translating findings into messages that help other 
patients make decisions about whether or not to participate in SMAs, 3) participating in conference presentations (co-
authoring and attending conferences), 4) contributing to framing and conclusions in published manuscripts (as co-
authors). Additionally, depending on the results, patient partners may advise adding or eliminating other features of 
SMAs, and may serve as advocates to the practices and payers to support maintaining the patient-driven SMA model.  
Similarly, practice and health plan stakeholder representatives will contribute to interpretation of findings from the 
practice and health system perspectives, and to dissemination in the peer-reviewed literature. They will help prepare 
briefs on the value of patient-driven versus standardized SMAs and other information relevant to potential adopting 
organizations such as personnel, space and scheduling requirements, reimbursement strategies, and related issues.  
Practice and health plan stakeholders will also ultimately determine whether to promote adoption of standardized or 
patient-driven SMAs, and to advise the research team on opportunities for further research on additional variations of 
SMAs and/or larger scale implementation and dissemination trials.  

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
22 
 
DISSEMINATION AND IMPLEMENTATION POTENTIAL  
For detailed instructions, refer to the Application Guidelines for your PFA. Do not exceed two pages.  
 
A. Describe the potential for disseminating and implementing the results of this research in other settings.  
Highly qualified researchers, innovative community partners, and patient and provider stakeholders experienced in 
intervention studies and program development worked closely together to develop the research questions and 
intervention to fit with existing clinical structures (e.g., SMAs) with minimal disruption and burden to primary care 
clinicians. Diabetes group visits are employed at the participating clinics (e.g., Denver Health had 858 SMAs with 223 
patients in a year), and incorporated into clinical flow and billing practices, thus providing reimbursement models.  
Novel to this project’s design, is the focus on the delivery of the SMAs, i.e., the comparison of the patient driven 
SMA model to a standardized SMA model while using the same TTIM curriculum. Patient stakeholders chose TTIM for 
diabetes education in the context of SMAs because (1) they resonated with the TTIM content, and (2) TTIM is flexible 
enough to be delivered in either manner (patient-centered vs. standardized). Both SMA models have potential benefits 
for the patients and providers/clinics. Although diabetes SMAs are accepted as a standard of care, they currently are not 
“standardized” in delivery. The proposed standardized SMA model would allow for both ease of dissemination as well as 
provide data about which SMA components lead to better outcomes which currently is a gap in the literature. The 
patient driven SMA model , conceptualized by the patient stakeholders, is designed for meeting patients “where they 
are at” in terms of their medical and psychosocial needs regarding diabetes self-management, and is augmented by peer 
support.  The patient driven SMA model may provide a relative advantage over the standardized SMA model in terms of 
patient engagement and clinical outcomes as it is customized for patients’ self-management needs. Thus, through the 
comparison of these two delivery models, the study will provide outcomes that allow practices to decide which delivery 
approach makes the most sense within their practices and for which patients.  
Dissemination plan. We will disseminate findings via messages and strategies tailored to key audiences, who have 
different information needs, preferences, and perspectives regarding the decisional dilemma of how and whether to 
offer or participate in diabetes SMAs (Table 7). Study practices will disseminate the results within their organizations, 
and in turn, these organizations will help disseminate results to patients and their communities, behavioral health and 
medical providers, health plans, state and national professional organizations. There is a key focus on stakeholder 
engagement (patients, providers, and health plans) in the dissemination process,  who will be invited to be co-authors 
on manuscripts, professional conference and community presentations, and in electronic media dissemination to hold 
true to the spirit of community based participatory research. The research team will also be available for consultation to 
other clinic sites who wish to implement the SMA models. 
Table 7. Dissemination message and strategy by audience 
Dissemination Message 
Dissemination Strategy 
Audience: Patients 
Messages: 1) Reasons to participate in SMAs, 2) 
Anticipated patient costs, and 3) Effect on patient 
centered outcomes (study results)  
x 
One page infographics.  Includes key patient messages about SMAs to 
be displayed in clinic waiting rooms, and on clinics’ websites.  
x 
In person, local meetings. Includes patient stakeholders, and local 
patient advisory boards from participating clinics and organizations  
Audience: Practices/Providers  
Messages: 1) This a way to engage patients and 
achieve patient centeredness, 2) Informs 
infrastructure/resources needs to deliver SMAs 
and to optimize quality of care, and 3) how to bill 
x 
Customized provider and patient infographics. Includes study results, 
patient/provider testimonials, potential impact for patients/providers  
x 
Development plan. For further spread of intervention within 
participating health care organizations 
Audience: Local and Regional Networks and Organizations, e.g., Colorado Community Health Network (primary care providers), 
Colorado Behavioral Health Council Integrated Care Advisory Board, Colorado Coalition for the Medically Underserved (CCMU), 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
23 
State Networks Of Colorado Ambulatory Practices and Partners (SNOCAP; see letter of support).  
Messages:  1) Use of SMAs to increase patient 
engagement and clinical outcomes, 2) Use of 
patient driven SMAs with a multidisciplinary team 
to enhance patient engagement and clinical 
outcomes (if our findings support this), and 3) 
Implications for policy, workforce development, 
integrated care and alternative payment models  
x 
Conference Presentations. Presentation materials developed and 
presented jointly by all research and implementation team members 
x 
Webinar. Via statewide webinar discuss findings and their implications  
x 
Media outlets. UCD’s Office of Media Relations and Jefferson Center’s 
Department of Marketing will assist in preparing print, broadcast (radio 
or television), online interviews, and press releases about our findings.  
x 
Websites. (e.g., UCD’s SNOCAP’s and Jefferson Center’s Office of 
Health Care Transformation websites, Facebook, and Twitter accounts  
Audience: Health Plans, e.g., Colorado Department of Healthcare Policy and Financing (HCPF) administers Medicaid for the State 
of Colorado, and Colorado Prevention Alliance (CPA) a collaborative of public and private health insurers (see letter of support) 
Messages: 1. Implications for payment 
methodologies for sustaining the intervention 
model, 2.Widespread dissemination of findings to 
their constituent providers and patient 
populations  
 
x 
Presentations to HCPF and CCMU describing results, patient and 
provider perspectives on the value to patients and resources required 
to deliver both SMA models 
x 
Briefs will be created to educate local, state, and federal policy makers, 
health care organizations, and patient-advocacy entities about the 
implications of findings for implementation and sustainability; and will  
describe resources required for SMAs’ set up in clinical practice, how to 
identify patients, and how to bill and document for SMAs  
Audience: National Organizations, e.g., the North American Primary Care Research Group (NAPCRG), AcademyHealth, SAMHSA-
HRSA Center for Integrated Health Solutions, the Collaborative Family Healthcare Association, American Association of Family 
Physicians, Agency for Heathcare Quality and Research, Peers for Progress and the American Diabetes Association 
Messages:  1) Effectiveness of SMAs to increase 
patient engagement and clinical outcomes, 2) 
Implementation Processes for  SMAs  and 3) 
Implications for policy, workforce development, 
integrated care, and alternative payment models   
x 
Peer reviewed manuscripts will be developed and submitted to inform 
other researchers, clinicians, professional organizations, and the public 
x 
Presentations will be submitted to national conferences 
x 
National organizations media will be contacted about our findings for 
further dissemination on their websites and at their conferences 
B. Describe possible barriers to disseminating and implementing the results of this research in other settings.  
Challenges to dissemination and implementation of this project are not unique to health care organizations that 
employ peer mentors and involved integrated care teams that work across primary care and mental health 
organizations, i.e., funding non-licensed peer mentors and care coordination and other integrated care team activities in 
a current health care system where funding streams for medical and behavioral health are bifurcated. Both the CU 
Department of Family Medicine and JCMH’s Office for Healthcare Transformation are heavily involved in clinical 
transformation and health care financing reform to support integrated care efforts and are working developing the 
business models for paying for currently nonbillable services. Also of note, is that the initial infrastructure set up and 
resources needed to implement the patient-driven SMA may be require additional startup costs for some clinics so it is 
important to demonstrate that this model shows significant clinical outcomes and patient engagement over a 
standardized SMA for a clinic’s population of patients with diabetes and/or a subset of this population that has 
significant psychosocial distress.   
C. Describe how you will make study results available to study participants after you complete your analyses.  
At enrollment, all study participants will be asked if they are interested in having study findings shared directly with 
them at the study conclusion. Interested participants will receive a summary of study results and will be notified of the 
possibility to engage in further dissemination activities if interested. Study findings will be provided as one page 
‘infographic briefs’, a plain language communication form that our department has pioneered and has substantial 
experience. The study team (including patient and provider stakeholders) will assure that the study findings will be 
communicated in a manner that insures relevance and understandability to study participants, including personal 
patient narratives to illustrate study results.  If we successfully receive further funding for dissemination activities, we 
will further partner with local public health departments, state organizations, healthcare consumer groups, to create 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
24 
written and web based communication strategies (e.g., webinars, social media).   
 
PROTECTION OF HUMAN SUBJECTS 
 
Describe the protection of human subjects involved in your research  
OVERVIEW 
This research will be undertaken with the strictest procedures in place to assure the safety, anonymity, and 
confidentiality of the research subjects involved.  This research protocol will be reviewed by the Colorado Multiple 
Institutional Review Board (COMIRB), the body that governs human subjects’ research at the University of Colorado 
Denver (UCD) and other local institutions. The study methods for the current proposal, including techniques for 
recruitment, conduct of a cluster randomized, pragmatic comparative effective trial of intervention strategies, reviews 
of existing administrative databases, and on-site data collection, are similar to methods used in prior IRB-approved 
protocols directed by the study team.  If practices involved in the study require their own institution’s IRB approval, 
these will also be obtained prior to starting any study procedures. Protection of human subjects applies to three parts of 
the proposed project: 1) qualitative interviews and surveys among providers and staff at participating practices; 2) 
qualitative interviews and out of pocket surveys among patients at participating practices; and 3) patient-level clinical 
and patient reported outcomes via analyses of local administrative datasets or electronic medical record data. The 
intervention (training and implementation of SMAs guided by the research team) is delivered at the practice-level, such 
that actual patient care (delivery of SMAs by local practice providers) is considered quality improvement. Thus, patients 
are not considered enrolled in research unless they are recruited for the interviews and out of pocket cost surveys.  
 
A. RISKS TO HUMAN SUBJECTS 
Human Subjects Involvement, Characteristics and Design: 
This study involves both primary data collection via qualitative and self-report surveys and a secondary analysis of 
existing clinical, demographic and administrative data from local practice electronic health records (“EHR data”) 
maintained by participating practices. Intervention implementation strategies informed and refined through focus 
groups and qualitative interviews will be developed and pilot-tested during the first year, implemented in years two and 
three, and assessed in year four. A cluster randomized, pragmatic comparative effective trial will be utilized, where 
practices will be assigned either to standardized or patient-driven SMA intervention groups to compare the 
effectiveness of the intervention strategies.  Patients in standardized groups will receive the standard intervention 
model at the participating sites.  Patients in patient-driven groups will receive enhanced SMAs as developed during year 
one.   
 
1. Practice Interviews: Selected practice staff and providers in participating practices will be asked to participate in 
key informant interviews to assess practice culture and context. These interviews will be completed at baseline 
and again at 9 and 24 months. 
2. Practice Surveys: All practice staff and providers in participating practices will be asked to complete survey 
measures assessing team-based care and quality of care. These surveys will be completed at baseline and again at 
9 and 24 months. 
3. Patient Interviews and out-of-pocket cost surveys: Patient interviews and out of pocket cost and time surveys will 
assess patient experience as well as patient-incurred costs of SMAs specifically and diabetes care more generally. 
Interviews will be conducted 35-40 purposefully selected patients sampled from each of the experimental 
conditions. 
4. Patient Reported Outcomes Measures: 
Patient surveys assessing outcome measures selected by stakeholders 
or consistent with the conceptual model will be completed at the first SMA session and the last SMA session, as 
part of clinical care. These data will be linked with clinical outcome data and stored in local data bases. 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
25 
5. Patient-level Clinical Outcomes data from the EHR: 
We will gather clinical outcomes from participating practices’ electronic health records (EHR). Diagnosis data from the 
EHR will be used to compute a comorbidity index and to corroborate diagnosis of mental illness. Prescription data from 
the EHR will be used to assess insulin dependence. Clinical outcomes data for each participating patient will be extracted 
from the participating practices’ electronic health records for the period 9 months before and after involvement in SMAs 
using chart review and electronic data extracts. 
 
Up to 1440 patients will be enrolled in either standardized or patient-driven SMAs and will contribute data to the 
analysis. Inclusion criteria for patients include adults at least 18 years old with Type II diabetes, and who receive care in 
participating practices. Exclusion criteria include pregnancy or plans to become pregnant in the next six months, limited 
cognitive capacity such as that due to dementia or a developmental disorder, less than one year life expectancy, or plans 
to leave the area in the next year. These criteria will be finalized with stakeholders.  Subgroups include patients with 
comorbid mental illness (depression, anxiety, bipolar disorder, psychotic disorders). Given the patient populations 
represented by the practice, this sample will include representation from a number of AHRQ priority groups: inner-city; 
low income; minority; women; children; elderly; and those who need chronic and/or mental health care.  
 
Sources of Materials  
Practice Surveys and Interviews: Practice staff and providers in participating practices will be asked to complete survey 
measures assessing aspects of practice context including measures of team based care (relational coordination) and 
quality of care (assessment of chronic illness care). Selected practice staff and providers involved in SMAs in 
participating practices will be asked to participate in key informant interviews, conducted by qualitative research team 
members using interview guides covering practice experiences with SMAs, including perceived value and sustainability, 
and to complete surveys assessing workflows and resource requirements to deliver SMAs. Interviews will be conducted 
by a trained and experienced qualitative interviewer. RedCap will be used to collect and manage survey data. Interviews 
will be recorded, transcribed and loaded into a qualitative analysis software program (Atlas.ti).  
Patient Cost Surveys and Interviews: A purposefully sampled subset of patients will also participate in qualitative 
interviews and complete cost surveys to assess patient experience and out of pocket costs of participating in SMAs 
specifically and diabetes care more generally. Patients will complete self-report surveys about cost of care. Interview 
guides and recorded interviews will support the qualitative data collection process. Interviews will be recorded, 
transcribed and loaded into a qualitative analysis software program (Atlas.ti). 
 
Patient Reported and Clinical Outcomes from the EHR: This study also involves secondary use of data from electronic 
health records and patient reported data which reside in local databases maintained by the participating practices. The 
data to be requested from the practice will be limited data sets, as they will include dates of services (for determining 
exposure and outcomes periods), but otherwise contain no direct identifiers, as defined by HIPAA. The patient reported 
outcome data are from surveys to be administered in the context of receiving care in SMAs, and include information 
about patient health behaviors, quality of life,  disease distress. The data from the EHR to be used for this study will 
include health, demographic and health care utilization data, including information on diagnoses, laboratory tests, vital 
signs, patient demographics, encounters and procedures. 
 
Potential Risks:  
Minimal risks to human subjects are anticipated for this study. There is a slight risk of psychological discomfort and/or a 
time burden associated with participating in qualitative interviews or completing the self-report surveys. There is also a 
slight risk of loss of confidentiality and/or anonymity. 
 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
26 
B. ADEQUACY OF PROTECTION AGAINST RISKS 
Informed Consent: 
For qualitative interviews with providers and staff: Each participant will receive an information consent (also known as 
“postcard consent” document, as approved by the University of Colorado’s Institutional Review Board; we will request a 
waiver of documentation of consent for these interviews. This will be presented in person by the study team and one 
copy of the informed consent document will remain with the subject. The consent form will inform participants that 
participation is voluntary, that they are free to withdraw from the study at any time, that all information is to be kept 
confidential, that their identity will not be revealed, that a code number will be used to track all of their information, 
and that only the research investigators will have access to the data. Written consent forms will be kept in a locked 
cabinet in the PI’s private, locked office.  All study results will be published and presented in aggregate form only, with 
no individual responses identified.  
 
For surveys with providers and staff: We will request a waiver of documentation of consent for participation in surveys, 
because the only risks to participants relate to a potential loss of confidentiality.  Any breach of confidentiality will be 
strictly guarded against, and any unintentional breach of confidentiality would not be anticipated to affect the 
employability, reputation, or financial status of a participant. We will use an information (or postcard) consent as the 
first page of practice surveys, informing respondents of the purpose of the surveys, confidentiality protections, and that 
completion is not-mandatory. All study results will be published and presented in aggregate form only, with no 
individual responses identified.  
 
For qualitative interviews and out of pocket cost surveys with patients: Each patient will receive an information consent 
document, as approved by the University of Colorado’s Institutional Review Board; we will request a waiver of 
documentation of consent. This will be presented in person by the study team and one copy of the information consent 
document will remain with the subject. The consent form will inform participants that participation is voluntary, that 
they are free to withdraw from the study at any time, that all information is to be kept confidential, that their identity 
will not be revealed, that a code number will be used to track all of their information, and that only the research 
investigators will have access to the data. Written consent forms will be kept in a locked cabinet in the PI’s private, 
locked office.  All study results will be published and presented in aggregate form only, with no individual responses 
identified. 
 
For surveys (non-cost related) and EHR data from patients: We will request a waiver of consent for linked survey and 
EHR data. These data will be transferred to the research team as a HIPAA-defined limited data set (including only birth 
dates, service dates and practice zip codes), with no direct identifiers. We will execute data use agreements with 
practices to use these data for evaluation of study outcomes.  
 
Protections Against Risk: 
The PI and select study personnel will not have access to individually identifiable health data. Patient survey data will be 
collected as part of their care, and linked with EHR data as described above. Investigators will not have access to contact 
information for potential participants unless potential participants voluntarily provide this information. Identifying 
information will not be linked to participant responses, such that all participant data will be confidential. The clinical 
data for this study will be requested from the EHR of the participating health care organization using established secure 
query protocols and query technology. Random identifiers will be assigned to patients who agree to participate; these 
random identifiers will be stored in the distributed databases to allow linkage of clinical and survey data. Required IRB 
approvals and data use agreements between the University of Colorado Denver and participating organizations will be 
obtained. Once the combined analytic data set from the surveys and the distributed database is created, it will be stored 
on a secure password-protected server in the Department of Family Medicine.  

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
27 
 
To mitigate risk of psychological discomfort and/or time burden, participants will be informed that they may choose not 
to complete any questions that make them uncomfortable and they may choose to withdraw from the study at any time 
without losing any benefits to which they may be entitled. To mitigate the risk of loss confidentiality and/or anonymity, 
the data set requested from the practice EHR for this research will contain the minimal amount of data required to 
conduct the proposed analyses, and will contain no direct identifiers other than a study-specific random identifier (to 
allow linkage with survey data) and only service dates as indirect identifiers. Secure web-based data transfer 
mechanisms using Egnyte.com will be used to transfer data from the local practice/organizational databases to a secure 
server at UCD. Data access will be limited to study personnel on a secure server at UCD, and all information will be kept 
confidential. This is likely to mitigate the risks identified. For all research groups the following additional protections will 
be implemented: 
x 
All investigators involved in the project have completed either the Program for Education and Evaluation in 
Responsible Research and Scholarship (PEERRS) or the Collaborative Institutional Training Initiative (CITI) online 
training courses related to human subject’s protections. These courses fulfill all NIH requirements for human 
subjects training. 
x 
Study activities involving human subjects will not begin until approval has been granted from all relevant 
Institutional Review Boards. Annual renewal of this approval will be required for continuation of the study. 
Although we do not anticipate any adverse events to occur in this study, should any occur, they would be reported to all 
involved IRBs in accordance with federal and institutional policies. 
 
C. POTENTIAL BENEFITS OF THE PROPOSED RESEARCH TO HUMAN SUBJECTS AND OTHERS 
There are no direct benefits to participants, with the possible exception of insight into one’s own health behaviors. 
However, their participation may benefit others, as this research is designed to inform more patient-centered and 
effective approaches to improving the health and well-being among those with Type II diabetes. The minimal risks (loss 
of confidentiality, psychological discomfort and time burden) to subjects are reasonable in relation to the potential 
benefits to the broader population. The risks are small and not serious, while benefits to the broader population in 
terms of reduced costs to society and improved health and well-being could be substantial.  
 
D. IMPORTANCE OF THE KNOWLEDGE TO BE GAINED 
Results from this study will significantly advance the understanding of the process of planning and implementing SMAs, 
including a curriculum that can be tailored to the diverse needs of their patient population. The findings can be 
extended to other populations, potentially having far reaching effects on implementation of SMAs among those with 
chronic conditions. If no differences in outcomes are demonstrated between standardized and patient-driven SMAs, this 
information is also important, as it points to a need to devise alternative strategies for connecting patients to evidence-
based services. The risks are small and not serious, while the knowledge to be gained has the potential to help support 
management of chronic disease and can improve quality of life. 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
28 
 
CONSORTIUM CONTRACTUAL ARRANGEMENTS  
 
 
Describe the proposed research projects that subcontracted organizations will perform. Explain the strengths that 
these partners bring to the overall project to ensure successful submission of contract deliverables in accordance with 
the milestone schedule. 
 
We will initiate subcontracts with the following organizations: 1) Jefferson Center for Mental Health; 2) Case Western 
Reserve University; and 3) Denver Health and Hospital Authority. 
 
Jefferson Center for Mental Health (JCMH) 
A subcontract will be established with JCMH to facilitate implementation of the intervention in participating clinics. 
JCMH, which has vast experience providing collocated and integrated behavioral health services at FQHCs and over 20 
private primary care practices will serve as the lead CMHC for this project. Funds will cover Dr. Waxmonsky as the Co-PI 
of the study. She will assure all aspects of implementation for the project, including stakeholder engagement, 
intervention implementation and fidelity monitoring, clinician training, data collection monitoring, and practice 
support/facilitation. In addition, we will pay for an implementation project manager to facilitate all day-to-day 
implementation activities, and for site leadership and staff to facilitate stakeholder engagement, establish and maintain 
research protocols, and provide behavioral health guidance throughout the project. 
 
Jefferson Center for Mental Health (JCMH) will serve as the lead CMHC for this project. JCMH has over 20 years’ 
experience providing collocated and integrated behavioral health services at FQHCs and over 20 private primary care 
practices.  Jefferson Center is unique in that it has an Office of Healthcare Transformation that provides both national 
and regionally consulting on healthcare policy, integrated care implementation and research. JCMH has a Wellness 
Program for all its patients and employs health coaches to assist in health behavior change. Additionally, JCMH has an 
existing diabetes/metabolic syndrome workgroup that works specifically with primary care clinician partners on 
addressing the needs of patients with diabetes and mental health disorders, and a cadre of peer specialists that can be 
deployed as peer mentors. JCMH and Aurora Mental Health Center (AuMHC), another community mental health center 
with a bi-directional health home, serve similar patient populations with medical illness and psychiatric comorbidities. 
 
Case Western Reserve University (CWRU) 
A subcontract will also be established with CWRU to conduct the training for the intervention. The subcontract funds will 
cover Dr. Sajatovic to assume the overall responsibility for use of the TTIM intervention, and to also participate in 
interpretation of study findings and to help facilitate disseminations of results as appropriate. The allotted funds will pay 
for Dr. Sajatovic’s time, 5% of a research manager, and travel expenses. 
 
Denver Health and Hospital Authority (DH) 
A subcontract will be established with DH to implement the intervention at two Denver Health Clinics.  DH is an 
academic medical center and is nationally recognized for its model of care delivery to underserved, indigent, and 
minority patients and for its growth and financial stability.  Funds will be used to cover Dr. Natalie Ritchie, who will 
oversee research implementation at those sites. Dr. Ritchie is a clinical health psychologist at DH and an Assistant 
Professor at the University of Colorado Denver, School of Medicine. She has subsequently served as the Principal 
Investigator (PI) of multiple awards to translate best practices in diabetes prevention and management to clinical 
settings serving diverse individuals. In addition to Dr. Ritchie’s time, this subcontract will also cover costs of a research 
manager, a statistical specialist, and travel expenses.

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
29 
 
REFERENCES CITED  
For detailed instructions, refer to the Application Guidelines for your PFA. Do not exceed 10 pages. 
 
Following scholarly citation practice (AMA style), list the source material cited in this Research Plan.  
1. 
Menke A, Casagrande S, Geiss L, Cowie CC. Prevalence of and Trends in Diabetes Among Adults in the United 
States, 1988-2012. Jama. Sep 8 2015;314(10):1021-1029. 
2. 
Guariguata L, Whiting DR, Hambleton I, Beagley J, Linnenkamp U, Shaw JE. Global estimates of diabetes 
prevalence for 2013 and projections for 2035. Diabetes Research and Clinical Practice.103(2):137-149. 
3. 
Standards of Medical Care in Diabetes-2016 Abridged for Primary Care Providers. Clin Diabetes. Jan 
2016;34(1):3-21. 
4. 
Standards of Medical Care in Diabetes-2016 Abridged for Primary Care Providers. Clinical diabetes : a publication 
of the American Diabetes Association. Jan 2016;34(1):3-21. 
5. 
Ali MK, Bullard KM, Saaddine JB, Cowie CC, Imperatore G, Gregg EW. Achievement of goals in U.S. diabetes care, 
1999-2010. N Engl J Med. Apr 25 2013;368(17):1613-1624. 
6. 
Boyle JP, Thompson TJ, Gregg EW, Barker LE, Williamson DF. Projection of the year 2050 burden of diabetes in 
the US adult population: dynamic modeling of incidence, mortality, and prediabetes prevalence. Popul Health 
Metr. 2010;8:29. 
7. 
Gregg EW, Li Y, Wang J, et al. Changes in diabetes-related complications in the United States, 1990-2010. N Engl 
J Med. Apr 17 2014;370(16):1514-1523. 
8. 
Li R, Barker LE, Shrestha S, et al. Changes over time in high out-of-pocket health care burden in U.S. adults with 
diabetes, 2001-2011. Diabetes Care. Jun 2014;37(6):1629-1635. 
9. 
Koopmanschap M. Coping with Type II diabetes: the patient's perspective. Diabetologia. Jul 2002;45(7):S18-22. 
10. 
Meyers JL, Parasuraman S, Bell KF, Graham JP, Candrilli SD. The high-cost, type 2 diabetes mellitus patient: an 
analysis of managed care administrative data. Arch Public Health. 2014;72(1):6. 
11. 
Egede LE, Zheng D, Simpson K. Comorbid depression is associated with increased health care use and 
expenditures in individuals with diabetes. Diabetes Care. Mar 2002;25(3):464-470. 
12. 
Rabi DM, Edwards AL, Southern DA, et al. Association of socio-economic status with diabetes prevalence and 
utilization of diabetes care services. BMC Health Services Research. 2006;6:124. 
13. 
Fisher L, Skaff MM, Mullan JT, et al. Clinical depression versus distress among patients with type 2 diabetes: not 
just a question of semantics. Diabetes Care. Mar 2007;30(3):542-548. 
14. 
Fisher L, Glasgow RE, Strycker LA. The relationship between diabetes distress and clinical depression with 
glycemic control among patients with type 2 diabetes. Diabetes Care. 2010;33(5):1034-1036. 
15. 
Inzucchi SE, Bergenstal RM, Buse JB, et al. Management of hyperglycemia in type 2 diabetes, 2015: a patient-
centered approach: update to a position statement of the American Diabetes Association and the European 
Association for the Study of Diabetes. Diabetes Care. Jan 2015;38(1):140-149. 
16. 
Nutting PA, Dickinson WP, Dickinson LM, et al. Use of chronic care model elements is associated with higher-
quality care for diabetes. Ann Fam Med. Jan-Feb 2007;5(1):14-20. 
17. 
Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the Chronic Care Model in the new millennium. Health 
Aff (Millwood). Jan-Feb 2009;28(1):75-85. 
18. 
Austin B, Wagner E, Hindmarsh M, Davis C. Elements of Effective Chronic Care: A Model for Optimizing 
Outcomes for the Chronically Ill. Epilepsy Behav. Aug 2000;1(4):S15-S20. 
19. 
Wagner EH. Chronic disease care. Bmj. Jan 24 2004;328(7433):177-178. 
20. 
Wagner EH, Grothaus LC, Sandhu N, et al. Chronic care clinics for diabetes in primary care: a system-wide 
randomized trial. Diabetes Care. Apr 2001;24(4):695-700. 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
30 
21. 
Norris SL, Lau J, Smith SJ, Schmid CH, Engelgau MM. Self-management education for adults with type 2 diabetes: 
a meta-analysis of the effect on glycemic control. Diabetes Care. Jul 2002;25(7):1159-1171. 
22. 
Haas L, Maryniuk M, Beck J, et al. National standards for diabetes self-management education and support. 
Diabetes Care. Jan 2013;36 Suppl 1:S100-108. 
23. 
Fisher L, Hessler D, Glasgow RE, et al. REDEEM: a pragmatic trial to reduce diabetes distress. Diabetes Care. Sep 
2013;36(9):2551-2558. 
24. 
Glasgow RE, Davis CL, Funnell MM, Beck A. Implementing practical interventions to support chronic illness self-
management. The Joint Commission Journal on Quality and Patient Safety. 2003;29(11):563-574. 
25. 
Edelman D, Gierisch JM, McDuffie JR, Oddone E, Williams JW, Jr. Shared medical appointments for patients with 
diabetes mellitus: a systematic review. J Gen Intern Med. Jan 2015;30(1):99-106. 
26. 
Healthcare Financial Managment Association. Shared Medical Appointments Improve Access, Quality, and 
Efficiency. 2013; http://www.hfma.org/Leadership/E-
Bulletins/2013/April/Shared_Medical_Appointments_Improve_Access,_Quality,_and_Efficiency/. Accessed 10 
Feb, 2016. 
27. 
Clinic C. Shared Medical Appointments.  http://my.clevelandclinic.org/professionals/shared-medical-
appointments. Accessed 10 Feb, 2016. 
28. 
Physicians AAoF. Shared Medical Appointments/Group Visits.  http://www.aafp.org/about/policies/all/shared-
medical.html. Accessed 10 Feb, 2016. 
29. 
Schillinger D, Grumbach K, Piette J, et al. Association of health literacy with diabetes outcomes. Jama. Jul 24-31 
2002;288(4):475-482. 
30. 
Sadur CN, Moline N, Costa M, et al. Diabetes management in a health maintenance organization. Efficacy of care 
management using cluster visits. Diabetes Care. Dec 1999;22(12):2011-2017. 
31. 
Blixen C, Perzynski A, Kanuch S, et al. Training peer educators to promote self-management skills in people with 
serious mental illness (SMI) and diabetes (DM) in a primary health care setting. Prim Health Care Res Dev. Apr 
2015;16(2):127-137. 
32. 
Tang TS, Funnell M, Sinco B, et al. Comparative effectiveness of peer leaders and community health workers in 
diabetes self-management support: results of a randomized controlled trial. Diabetes Care. Jun 2014;37(6):1525-
1534. 
33. 
Heisler M. Different models to mobilize peer support to improve diabetes self-management and clinical 
outcomes: evidence, logistics, evaluation considerations and needs for future research. Fam Pract. Jun 2010;27 
Suppl 1:i23-32. 
34. 
Brownson CA, Heisler M. The role of peer support in diabetes care and self-management. Patient. Mar 1 
2009;2(1):5-17. 
35. 
Wagner EH, Bennett SM, Austin BT, Greene SM, Schaefer JK, Vonkorff M. Finding common ground: patient-
centeredness and evidence-based chronic illness care. J Altern Complement Med. 2005;11 Suppl 1:S7-15. 
36. 
Romeo GR, Abrahamson MJ. The 2015 standards for diabetes care: maintaining a patient-centered approach. 
Ann Intern Med. Jun 2 2015;162(11):785-786. 
37. 
Kwan BM, Sills MR, Graham D, et al. Stakeholder Engagement in a Patient-Reported Outcomes (PRO) Measure 
Implementation: A Report from the SAFTINet Practice-based Research Network (PBRN). J Am Board Fam Med. 
Jan-Feb 2016;29(1):102-115. 
38. 
Norman N, Bennett C, Cowart S, et al. Boot camp translation: a method for building a community of solution. J 
Am Board Fam Med. May-Jun 2013;26(3):254-263. 
39. 
Kwan BM, Jortberg B, Warman MK, et al. Stakeholder Engagement in Diabetes Self-Management:Patient 
Preference for Peer Support and Other Insights. Family Practice: Intl J of Res Prim Care. in press. 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
31 
40. 
Yost KJ, Webster K, Baker DW, Choi SW, Bode RK, Hahn EA. Bilingual health literacy assessment using the Talking 
Touchscreen/la Pantalla Parlanchina: Development and pilot testing. Patient Educ Couns. Jun 2009;75(3):295-
301. 
41. 
Lawless ME, Kanuch SW, Martin S, et al. A Nursing Approach to Self-Management Education for Individuals With 
Mental Illness and Diabetes. Diabetes Spectrum. 2016;29(1):24-31. 
42. 
Sajatovic M, Dawson NV, Perzynski AT, et al. Best practices: Optimizing care for people with serious mental 
illness and comorbid diabetes. Psychiatric Services. Sep 2011;62(9):1001-1003. 
43. 
Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based interventions in health 
care: application of the replicating effective programs framework. Implementation Science. 2007;2(1):1. 
44. 
Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-
AIM framework. Am J Public Health. Sep 1999;89(9):1322-1327. 
45. 
Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and 
well-being. Am Psychol. Jan 2000;55(1):68-78. 
46. 
Mead N, Bower P. Patient-centredness: a conceptual framework and review of the empirical literature. Soc Sci 
Med. Oct 2000;51(7):1087-1110. 
47. 
Williams GC, Patrick H, Niemiec CP, et al. Reducing the health risks of diabetes: how self-determination theory 
may help improve medication adherence and quality of life. Diabetes Educ. May-Jun 2009;35(3):484-492. 
48. 
Teixeira PJ, Carraca EV, Markland D, Silva MN, Ryan RM. Exercise, physical activity, and self-determination 
theory: a systematic review. Int J Behav Nutr Phys Act. 2012;9:78. 
49. 
Williams GC, McGregor HA, Zeldman A, Freedman ZR, Deci EL. Testing a self-determination theory process 
model for promoting glycemic control through diabetes self-management. Health Psychology. 2004;23(1):58. 
50. 
Williams GC, Lynch M, Glasgow RE. Computer-assisted intervention improves patient-centered diabetes care by 
increasing autonomy support. Health Psychology. 2007;26(6):728. 
51. 
Deverka PA, Lavallee DC, Desai PJ, et al. Stakeholder participation in comparative effectiveness research: 
defining a framework for effective engagement. J Comp Eff Res. Mar 2012;1(2):181-194. 
52. 
Glasgow RE LL. Evaluation of theory-based interventions. . In: Glanz K RB, Viswanath K, ed. Health Behavior and 
Health Education: Theory, Research, and Practice. 4th ed. ed. San Francisco: Jossey-Bass; 2008:487-508. 
53. 
Kessler R, Glasgow RE. A proposal to speed translation of healthcare research into practice: dramatic change is 
needed. Am J Prev Med. Jun 2011;40(6):637-644. 
54. 
Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public 
Health. Jun 2013;103(6):e38-46. 
55. 
Li F, Lokhnygina Y, Murray DM, Heagerty PJ, DeLong ER. An evaluation of constrained randomization for the 
design and analysis of group-randomized trials. Stat Med. 2015. 
56. 
Moulton LH. Covariate-based constrained randomization of group-randomized trials. Clinical Trials. 
2004;1(3):297-305. 
57. 
Dickinson LM, Beaty B, Fox C, et al. Pragmatic Cluster Randomized Trials Using Covariate Constrained 
Randomization: A Method for Practice-based Research Networks (PBRNs). J Am Board Fam Med. Sep-Oct 
2015;28(5):663-672. 
58. 
Hedeker D, Gibbons RD, Waternaux C. Sample size estimation for longitudinal designs with attrition: comparing 
time-related contrasts between two groups. Journal of Educational and Behavioral Statistics. 1999;24(1):70-93. 
59. 
Taveira TH, Dooley AG, Cohen LB, Khatana SA, Wu WC. Pharmacist-led group medical appointments for the 
management of type 2 diabetes with comorbid depression in older adults. Ann Pharmacother. Nov 
2011;45(11):1346-1355. 
60. 
Fisher L, Hessler DM, Polonsky WH, Mullan J. When is diabetes distress clinically meaningful?: establishing cut 
points for the Diabetes Distress Scale. Diabetes Care. Feb 2012;35(2):259-264. 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
32 
61. 
Polonsky WH, Fisher L, Earles J, et al. Assessing psychosocial distress in diabetes: development of the diabetes 
distress scale. Diabetes Care. Mar 2005;28(3):626-631. 
62. 
Dickinson WP, Dickinson LM, Nutting PA, et al. Practice facilitation to improve diabetes care in primary care: a 
report from the EPIC randomized clinical trial. Ann Fam Med. Jan-Feb 2014;12(1):8-16. 
63. 
Norris SL, Chowdhury FM, Van Le K, et al. Effectiveness of community health workers in the care of persons with 
diabetes. Diabet Med. May 2006;23(5):544-556. 
64. 
Heisler M, Choi H, Palmisano G, et al. Comparison of community health worker-led diabetes medication 
decision-making support for low-income Latino and African American adults with diabetes using e-health tools 
versus print materials: a randomized, controlled trial. Ann Intern Med. Nov 18 2014;161(10 Suppl):S13-22. 
65. 
Rowe AK, de Savigny D, Lanata CF, Victora CG. How can we achieve and maintain high-quality performance of 
health workers in low-resource settings? The lancet. 2005;366(9490):1026-1035. 
66. 
Hodorowicz MA. Reimbursement for Shared Medical Appointments Incorporating Diabetes Self-Management 
Education/Training or Diabetes Medical Nutrition Therapy. Diabetes Spectrum. May 1, 2012 2012;25(2):84-90. 
67. 
Physicians AAoF. Coding for Group Visits.  http://www.aafp.org/practice-management/payment/coding/group-
visits.html. Accessed 10 Feb, 2016. 
68. 
Dickinson LM, Beaty B, Fox C, et al. Pragmatic Cluster Randomized Trials Using Covariate Constrained 
Randomization: A Method for Practice-based Research Networks (PBRNs). Journal of the American Board of 
Family Medicine. Sep-Oct 2015;28(5):663-+. 
69. 
Moulton LH. Covariate-based constrained randomization of group-randomized trials. Clinical trials (London, 
England). 2004;1(3):297-305. 
70. 
SAS V 9.3 [computer program]. Cary, NC: SAS Institute, Inc. 
71. 
Kirchner JE, Ritchie MJ, Pitcock JA, Parker LE, Curran GM, Fortney JC. Outcomes of a partnered facilitation 
strategy to implement primary care-mental health. J Gen Intern Med. Dec 2014;29 Suppl 4:904-912. 
72. 
Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D 
(EQ-5D-5L). Qual Life Res. Dec 2011;20(10):1727-1736. 
73. 
Williams GC, Freedman ZR, Deci EL. Supporting autonomy to motivate patients with diabetes for glucose control. 
Diabetes Care. 1998;21(10):1644-1651. 
74. 
Toobert DJ, Hampson SE, Glasgow RE. The summary of diabetes self-care activities measure: results from 7 
studies and a revised scale. Diabetes Care. Jul 2000;23(7):943-950. 
75. 
Hwang W, Weller W, Ireys H, Anderson G. Out-of-pocket medical spending for care of chronic conditions. Health 
Aff (Millwood). 2001;20(6):267-278. 
76. 
Russell LB, Ibuka Y, Carr D. How much time do patients spend on outpatient visits? The Patient: Patient-Centered 
Outcomes Research. 2008;1(3):211-222. 
77. 
Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative 
databases. J Clin Epidemiol. Jun 1992;45(6):613-619. 
78. 
Raue PJ, Sirey JA. Designing personalized treatment engagement interventions for depressed older adults. 
Psychiatric Clinics of North America. 2011;34(2):489-500. 
79. 
Dickinson LM, Dickinson WP, Nutting PA, et al. Practice context affects efforts to improve diabetes care for 
primary care patients: a pragmatic cluster randomized trial. J Gen Intern Med. Apr 2015;30(4):476-482. 
80. 
Gittell JH, Godfrey M, Thistlethwaite J. Interprofessional collaborative practice and relational coordination: 
improving healthcare through relationships. J Interprof Care. May 2013;27(3):210-213. 
81. 
Noel PH, Lanham HJ, Palmer RF, Leykum LK, Parchman ML. The importance of relational coordination and 
reciprocal learning for chronic illness care within primary care teams. Health Care Manage Rev. Jan-Mar 
2013;38(1):20-28. 
82. 
Bonomi AE, Wagner EH, Glasgow RE, VonKorff M. Assessment of chronic illness care (ACIC): a practical tool to 
measure quality improvement. Health Serv Res. Jun 2002;37(3):791-820. 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
33 
83. 
Kaplan RS, Haas DA, Warsh J. Adding Value by Talking More. New England Journal of Medicine. 
2016;375(20):1918-1920. 
84. 
Schilling LM, Kwan BM, Drolshagen CT, et al. Scalable Architecture for Federated Translational Inquiries Network 
(SAFTINet) technology infrastructure for a distributed data network. eGEMs (Generating Evidence & Methods to 
improve patient outcomes). 2013;1(1):11. 
85. 
Brown J, Kahn M, Toh S. Data quality assessment for comparative effectiveness research in distributed data 
networks. Med Care. 2013;51(8 0 3):S22. 
86. 
Gilchrist V, Williams R. Key Informant Interviews. In: Crabtree B, Miller W, eds. Doing Qualitative Research. 2 ed. 
Thousand Oaks, CA: Sage; 1999:71-88. 
87. 
Potworowski G, Green L. Cognitive task analysis: methods to improve patient-centered medical home models by 
understanding and leveraging its knowledge work. Rockville MD: Agency for Healthcare Research and Quality. 
2013. 
88. 
Wright N, Ivers N, Eldridge S, Taljaard M, Bremner S. A review of the use of covariates in cluster randomized 
trials uncovers marked discrepancies between guidance and practice. J Clin Epidemiol. 2015;68(6):603-609. 
89. 
Murray DM. Design and analysis of group-randomized trials. Vol 29: Oxford University Press, USA; 1998. 
90. 
Littell R, Milliken GA, Stroup WW, Wolfinger RD, eds. SAS System for Mixed Models. Cary, NC: SAS Institute Inc; 
1996. 
91. 
Hedeker D, Gibbons RD. Longitudinal Data Analysis. Hoboken, New Jersey: John Wiley & Sons; 2006. 
92. 
Raman R, Hedeker D. A mixed-effects regression model for three-level ordinal response data. Stat Med. Nov 15 
2005;24(21):3331-3345. 
93. 
Bryk  AS, Raudenbush SW, eds. Hierarchical Linear Models: Applications and Data Analysis Methods. Second ed. 
Newbury Park: Sage Publications; 2000. 
94. 
Murray D, ed Design and analysis of group-randomized trials. New York: Oxford University Press; 1998. 
95. 
Giraudeau B, Ravaud P. Preventing Bias in Cluster Randomised Trials. PLoS Med. 2009;6(5):e1000065. 
96. 
Puffer S, Torgerson D, Watson J. Evidence for risk of bias in cluster randomised trials: Review of recent trials 
published in three general medical journals. BMJ. 2003 Oct 4 2003;327(7418):785-789. 
97. 
Campbell MK, Elbourne DR, Altman DG. CONSORT statement: extension to cluster randomised trials. BMJ. 
March 20, 2004 2004;328(7441):702-708. 
98. 
AP D, NM L, DB R. Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the 
Royal Statistical Society: Series B 1977;39:1-38. 
99. 
P D, MG K. Informative drop-out in longitudinal data analysis. Appl Stat 1994;43:49-93. 
100. 
RJA L. Statistical analysis with missing data. New York: Wiley; 1987. 
101. 
DL F. Design and analysis of quality of life studies in clinical trials. New York: Chapman & Hall/CRC; 2010. 
102. 
Hedeker D, Gibbons RD. Application of random-effects pattern-mixture models for missing data in longitudinal 
studies. Psychological Methods. Mar 1997;2(1):64-78. 
103. 
Littell RC, Stroup WW, Milliken GA, Wolfinger RD, Schabenberger O. SAS for mixed models: SAS institute; 2006. 
104. 
Dempster A, Laird N, Rubin D. Maximum likelihood estimation from incomplete data. Journal of the Royal 
Statistical Society (B). 1977;39(1):1-38. 
105. 
Little RJ, Rubin DB. Statistical analysis with missing data: John Wiley & Sons; 2014. 
106. 
Diggle P, Kenward MG. Informative drop-out in longitudinal data analysis. Applied statistics. 1994:49-93. 
107. 
Fairclough DL. Design and analysis of quality of life studies in clinical trials: CRC press; 2010. 
108. 
Liu L, Lee MJ, Brateanu A. Improved A1C and Lipid Profile in Patients Referred to Diabetes Education Programs in 
a Wide Health Care Network: A Retrospective Study. Diabetes spectrum : a publication of the American Diabetes 
Association. Nov 2014;27(4):297-303. 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
34 
109. 
Hawkins J, Kieffer EC, Sinco B, Spencer M, Anderson M, Rosland AM. Does Gender Influence Participation? 
Predictors of Participation in a Community Health Worker Diabetes Management Intervention With African 
American and Latino Adults. Diabetes Educator. Sep 2013;39(5):647-654. 
110. 
Addison R. A Grounded Hermenuetic Editing Approach. In: Crabtree B, Miller W, eds. Doing Qualitative 
Research. 2nd ed. Thousand Oaks, CA: Sage; 1999:145-161. 
111. 
B R, CC R. Configurational comparative methods: qualiative comparative analysis (QCA) and related techniques. . 
Thousand Oaks, CA: Sage; 2009. 
112. 
C R. Studying cases as configurations. In: C R, ed. Fuzzy-set social science. Chicago: University of Chicago Press; 
2000:64-87. 
113. 
C R. Causal complexity. In: C R, ed. Fuzzy-set social science. Chicago: University of Chicago Press; 2000:88-119. 
114. 
JS H, G P, C R, LA G. Necessary and sufficient conditions for high quality care management: an analysis using 
qualitative comparative analysis (QCA). . under review. 2016. 
115. 
Kessler R, Glasgow RE. A proposal to speed translation of healthcare research into practice: dramatic change is 
needed. Am J Prev Med. 2011;40(6):637-644. 
116. 
Campbell MK, Elbourne DR, Altman DG. CONSORT statement: extension to cluster randomised trials. Bmj. Mar 
20 2004;328(7441):702-708. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
35 
 
APPENDIX (optional)  
 
 
Appendix A. TTIM Curriculum Modules  
 
TTIM Curriculum Modules 
 
Module 
Menu of Topic(s) 
Core 
Diabetes/Mental 
Health 
1 
Orientation and introductions, Emphasize ground rules, Establishment of a therapeutic 
relationship, An introduction to DM, An introduction to personal goal-setting 
Core (always 1st) 
2 
Diabetes complications and benefits of change, Blood sugar monitoring, Symptoms of 
high/low blood sugar 
Core  
3 
The challenge of having both mental health conditions (MHC) and DM, Stigma of MHC and 
strategies to cope with stigma, Relationship of MHC symptoms and functioning in response 
to stress and DM 
Mental Health 
4 
Personal MHC profile (what does worsening illness look like for you), Triggers of MHC 
relapse, Personal action plan for coping with MHC relapse 
Mental Health 
5 
Problem-solving skills and the IDEA approach (Identify the problem, Define possible 
solutions, Evaluate the solutions, Act on the best solution), Talking with your medical 
providers, Role play of communication with care providers 
Core 
6 
Nutrition for best physical and emotional health, Reading labels  
Core 
7 
Replacing unhealthy sugar and fat, Substance use and its effects on DM, Problem-solving to 
feed your body healthfully 
Core 
8 
Effects of exercise on physical and emotional health, The importance of daily routine and 
good sleep habits 
Core 
9 
Medications and psychological treatments for MHCs, A personal care plan to take care of 
the mind & body; Talking with your mental health providers 
Mental Health 
10 
Social supports and using your available supports, Types of physical activity and your 
community  
Core 
11 
Taking care of your feet, Staying on track with medication treatments  
Core 
12 
Illness management as a life-style, Acknowledgement of group progress, Setting the stage 
for ongoing Illness management and recovery (Step 2) 
Core (always last) 
 
 
 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
36 
Appendix B. Patient-Driven Shared Medical Appointments Implementation Planning Worksheet Outline 
 
INSTRUCTIONS AND EXPECTATIONS: 
This instrument should be completed by appropriate organization personnel in collaboration with project personnel.  
This is a working document to facilitate the integration and implementation of diabetes SMAs. Please read through this 
entire document before completing any questions, to ensure full understanding of research team expectations and the 
decisions to be made by your organization. 
 
 
Organization Name:        
 
 
Date completed: 
      
 
Person(s) completing the worksheet:        
 
 
1. Identify the patient population 
a. What are their characteristics? (i.e., eligibility, priority) 
b. How will they be identified? (i.e., registries, systematic assessment) 
c. Who will identify them? 
d. When will they be identified? 
2. Identify local practice and organizational resources needed 
a. SMA health educator 
b. SMA peer mentor 
c. Other SMA care team members 
d. SMA coordinator  
e. Physical space 
f. Information Technology 
g. Quality Improvement 
3. Training 
a. Key staff training 
b. Timelines for rollout 
4. Recruit or invite eligible patients 
a. What is the message to eligible patients? 
b. Who invites eligible patients? 
5. Scheduling SMAs 
a. Where and when will SMAs be held? 
b. Who schedules and in what system? 
6. Data collection 
a. Patient assessments 
b. Tracking invitations, acceptance and participation by patients 
c. Tracking elements of modular curriculum delivered (fidelity) 
d. Tracking connections to the care team (communication/consultation and referral) 
7. Incorporate into shared care plan 
a. Care plan location (EMR) 
b. What is the process/ technology used for sharing of care plan among all care team members?  

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
37 
8. Preparation for the Intervention (SMAs) 
a. Staffing: Who delivers the SMAs? 
b. Where are they delivered? 
c. What are the administrative and clinical/agency requirements? 
d. How is the intervention promoted throughout the clinic?  
9. Keeping the Intervention Going 
a. How does the clinic team address staff turnover? 
b. How does the research (implementation) team address clinical team’s concerns? 
c. How does the clinic team receive technical assistance in adapting SMAs to their setting? 
10. Keeping patients engaged 
a. How are patients outreached by clinical staff?  
b. How are patients’ concerns addressed?  
11. Changing and Sustaining Clinic Culture to Support SMA 
a. How will barriers to implementation, clinical staff concerns be addressed? 
b. How will patient (and provider) successes be shared with clinic staff? 
c. How can the clinic staff be incentivized? 
12. Sustainability 
a. Who are key stakeholders for sustainability outside of the clinic?  
b. How are these key stakeholders engaged in the sustainability discussion? 
c. What data is needed to inform the business case for sustainability?    
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
38 
Appendix C. TTIM Fidelity Checklist 
Intervention Fidelity Evaluation Form: 
 
TTIM will be assessed with respect to fidelity (how closely Nurse interventionists follow the treatment manual).   
Fidelity to TTIM processes, content, and format will be evaluated by non-interventionist study staff randomly attending 
20% of TTIM sessions to determine if: 
1. Sessions covered relevant TTIM constructs and health practices,  
2. Format as appropriate, and  
3. If sufficient time was devoted to the question/answer/comment session. 
 
FIDELITY CHECK FACE-SHEET: 
 
DATE: 
RATER INITIALS: 
COHORT #:____ 
SESSION #:____ 
 
TTIM Fidelity Check SESSION 1: 
 
Topics covered: 
Orientation and introductions: Yes__  No__  
Emphasize ground rules: Yes__  No__ 
Discuss facts and misconceptions about SMI Yes__  No__ 
An introduction to DM: Yes__  No__ 
Did the interventionist establishment groundwork for a therapeutic relationship? (non-judgemental, collaborative and 
supportive: Yes__  No__   
 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
TTIM Fidelity Check SESSION 2: 
 
Topics Covered: 
The challenge of having both SMI and DM Yes__  No__ 
Stigma of SMI and strategies to cope with stigma Yes__  No__ 
Relationship of SMI symptoms and functioning in response to stress and DM Yes__  No__ 
An introduction to personal goal-setting Yes__  No__ 
 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
TTIM Fidelity Check SESSION 3: 
Topics Covered: 
 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
39 
Personal SMI profile (what does worsening illness look like for you Yes__  No__ 
Triggers of SMI relapse Yes__  No__ 
Personal action plan for coping with SMI relapse Yes__  No__ 
 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
 
TTIM Fidelity Check SESSION 4: 
Topics Covered: 
Diabetes complications and benefits of change Yes__  No__ 
Blood sugar monitoring Yes__  No__ 
Symptoms of high/low blood sugar Yes__  No__ 
 
Format: 
60-90 minutes in duration: Yes__  No__ 
 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
TTIM Fidelity Check SESSION 5: 
Topics Covered: 
Problem-solving skills and the IDEA approach Yes__  No__ 
Talking with your medical and your mental health care providers Yes__  No__ 
Role play of communication with care providers Yes__  No__ 
 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
TTIM Fidelity Check SESSION 6: 
 
Topics Covered: 
Nutrition for best physical and emotional health Yes__  No__ 
Reading labels Yes__  No__ 
 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
TTIM Fidelity Check SESSION 7: 
Topics Covered: 
Replacing unhealthy sugar and fat Yes__  No__ 
Substance use and its effects on SMI and on DM Yes__  No__ 
Problem-solving to feed your body healthfully Yes__  No__ 
 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
40 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
TTIM Fidelity Check SESSION 8: 
Topics Covered: 
Effects of exercise on physical and emotional health Yes__  No__ 
The importance of daily routine and good sleep habits Yes__  No__ 
 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
TTIM Fidelity Check SESSION 9: 
Topics Covered: 
Medications and psychological treatments for SMI Yes__  No__ 
A personal care plan to take care of the mind & body Yes__  No__ 
 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
TTIM Fidelity Check SESSION 10: 
Topics Covered: 
Social supports and using your available supports Yes__  No__ 
Types of physical activity and your community Yes__  No__ 
 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
TTIM Fidelity Check SESSION 11: 
Topics Covered: 
Taking care of your feet Yes__  No__ 
Staying on track with medication treatments Yes__  No__ 
 
Format: 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
TTIM Fidelity Check SESSION 12: 
Topics Covered: 
Illness management as a life-style Yes__  No__  
Acknowledgement of group progress Yes__  No__ 
Setting the stage for ongoing Illness management and recovery (Step 2) Yes__  No__ 
 
Format: 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
41 
60-90 minutes in duration: Yes__  No__ 
Sufficient time to answer questions that arose during group process: Yes__  No__ 
 
 
 

 
 
 
PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews 
 
 
 
PCORI Cycle 3 2016 Research Plan Template                                                                        
42 
Appendix D. Snapsot of TTIM Treatment Manual including scripts for care team members 
Part One: Having a Mental Illness and Diabetes Together: 
Nurse Educator script (Poster 4):  Review and discuss Poster #4. 
 “People with SMI are more likely to have diabetes than people without SMI.  At the same time, people with 
diabetes are more likely to have SMI than to those who do not have diabetes.” 
 
“People with SMI and Diabetes have medical and mental health problems.  Examples of this are higher rates of 
heart attack and psychiatric hospitalization.”  So, if SMI and Diabetes are not in good control, it could effect 
how long a person lives.   
 
“Studies show that taking care of Diabetes has good effects on mental health and that taking care of your 
mental health has good effects on Diabetes. So you can have a healthier and longer life by taking care of your 
SMI AND your Diabetes.” 
 
“Taking care of SMI includes taking prescribed medications, following up with counseling/case management, 
avoiding drugs and alcohol, having daily routines such as a regular time to go to sleep and wake up, as well as 
taking advantage of supports like family and friends.” 
 
“Taking care of Diabetes includes following a proper diet, exercising, taking prescribed medications and testing 
blood sugars as recommended. 
Part Two: Stigma of SMI and Strategies to Cope with Stigma: 
 
Nurse Interventionist Script: 
“We are now going to talk about STIGMA.  This follows the discussion we had last session on “myths” about 
mental illness. 
 
Our feelings and knowledge about mental illness are influenced by what we have learned from other people. 
One of the difficulties that some people encounter is psychiatric stigma. Psychiatric stigma is caused by the 
inaccurate beliefs, stereotypes, myths, and misconceptions people have about mental illness. Stigma influences 
the way others act toward those people who have mental illness. Stigma also affects the way people feel about 
having mental illness and can even affect a person’s behavior. For example, it may make a person less likely to 
take their medication. Have these comments affected the way that you think, feel, or manage your condition?” 
 
What might be some good ways to deal with cope with STIGMA? 
 
Note to Leaders: Discuss ways to cope with Stigma (Poster 5).  Review and discuss Poster #5. 
Give door prizes for additional, first/best answers on ways to cope with Stigma. 
 
 
 
 
 

PRINCIPAL INVESTIGATOR (LAST, FIRST, MIDDLE): Kwan, Bethany Matthews
PCORI Cycle 3 2016 Research Plan Template                                                                       
43
Part Three: Relationship of Mental Symptoms and Functioning in Response to Stress and Diabetes:
Nurse Educator Script:
“Everybody has stress and everyone in this room (Including your group leaders!!) is likely to have some future 
stress in their life. People respond in different ways when they are stressed or their SMI symptoms are getting 
worse (relapse).  This behavior can be called a “coping response”.  Sometimes the coping response has a good 
outcome.  Other times the outcome can have no effect or be negative/bad.  
Some coping responses can worsen Diabetes and mental state.  Sometimes people will try one coping response 
for a while, and then switch to another behavior if the first one does not work.  Sometimes people use more 
than one coping response when they are stressed.”
Poster #6 shows some of the coping responses that people can have in response to stress.  Think about how 
these coping responses might have good/neutral or effects on both mental illness and on Diabetes (Poster 6).
Review and discuss Poster #6.
Questions for the group:
1. Has any one ever used any of these in the past to deal with stress or symptoms of mental illness?
2. What kinds of coping responses can either worsen or improve Diabetes?
3. What would be coping responses to stress that could help BOTH mental illness and Diabetes?
Note to Leaders: 
Start with the list of possible coping responses and possible outcomes of these actions.  Encourage group 
members to generate additional coping responses and focus on identifying coping responses that can have 
either positive or at least neutral effects on both SMI and Diabetes.  Give door prizes to first/best responses on 
coping responses for management of both SMI and Diabetes.
Conclude this segment by pointing out that learning to cope with stress requires both practice and patience.  
By thinking about coping responses that are likely to be helpful in advance, individuals might be able to plan 
and possible be able to use these ideas for future stress.
You Answer: Coping responses that could help BOTH mental illness and 
Diabetes:
________________________________________________________________
________________________________________________________________
________________________________________________________________