Comparing Two Approaches to Care Coordination for High-Cost/High-Need Patients in Primary Care
Conditions: Chronic Disease, Multi-morbidity, Care Coordination
Sponsor: HealthPartners Institute

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
1  
 
 
 
 
 
 
 
 
 
 
Complete Title: Comparing Two Approaches to Care Coordination for High-Cost/High-
Need Patients in Primary Care 
 
 
 
 
 
Short Title: Minnesota Care CooRdination Effectiveness Study (MNCARES)  
 
Study Sponsor: Patient Centered Outcomes Research Institute (PCORI)  
PCORI Program Officer: Tabassum Majid, PhD 
Study Contract Number: IHS-2019C1-15625  
 
Principal Investigators: Leif Solberg, MD and Steven Dehmer, PhD 
Institution: HealthPartners Institute 
 
Protocol Date:  January 29, 2024 
Version  1.3 
 
 
 
 
 
 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
2  
 
Table of Contents 
Table of Contents .......................................................................................................................................... 2 
List of Abbreviations ..................................................................................................................................... 5 
Protocol Summary ........................................................................................................................................ 6 
Key Roles and Contact Information .............................................................................................................. 7 
Study Protocol ............................................................................................................................................... 8 
1 
Introduction: Background and Scientific Rationale .......................................................................... 8 
1.1 
Background ............................................................................................................................... 8 
1.2 
Evidence Gaps in Care Coordination for High-cost/High-need Patients ................................... 8 
1.3 
How this study addresses evidence gaps .................................................................................. 9 
1.4 
Health Care Homes in Minnesota ............................................................................................. 9 
2 
Study Aims and Outcomes .............................................................................................................. 10 
2.1 
Study Aims .............................................................................................................................. 10 
Figure 1: Timing of Patient Cohort and Survey Data Collection Efforts ...................................................... 10 
2.2 
Study Comparators (SC-1) ....................................................................................................... 11 
Table 1: Strategies for Measuring and Accounting for Fidelity to Each Care Coordination Model ........ 12 
2.3 
Study Outcomes ...................................................................................................................... 13 
3 
Study Design.................................................................................................................................... 13 
3.1 
Conceptual Framework ........................................................................................................... 14 
Figure 2: Care Coordination Measurement Framework ................................ Error! Bookmark not defined. 
3.2 
How and why the study sites were selected........................................................................... 14 
4 
Study Population and Recruitment ................................................................................................. 15 
4.1 
Setting ..................................................................................................................................... 15 
4.2 
Inclusion Criteria ..................................................................................................................... 15 
4.3 
Exclusion Criteria ..................................................................................................................... 16 
4.4 
Recruitment and Retention Strategies ................................................................................... 16 
4.5 
Clinic Participant Withdrawal ................................................................................................. 19 
5 
Data Collection Procedures ............................................................................................................. 20 
5.1 
Care Quality, Utilization, and Patient Reported Outcomes .................................................... 20 
Figure 3: Outcome Data Collection and Management ............................................................................... 20 
5.2 
Organizational Survey ............................................................................................................. 22 
5.3 
Semi-Structured Individual Interviews and Group Discussions .............................................. 23 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
3  
 
5.4 
Patient Surveys (Historical and Primary Cohorts) ................................................................... 27 
5.5 
Care Coordinator Survey ......................................................................................................... 29 
5.6 
Independent Variables, Measured Potential Confounders .................................................... 30 
Table 2: Independent Variables .................................................................................................................. 31 
6 
Data Management Plan .................................................................................................................. 31 
6.1 
Data collection ........................................................................................................................ 31 
6.2 
Data organization .................................................................................................................... 32 
6.3 
Data handling .......................................................................................................................... 32 
6.4 
Data documentation ............................................................................................................... 34 
6.5 
Data storage and preservation ............................................................................................... 34 
6.6 
Data maintenance ................................................................................................................... 34 
6.7 
Data sharing ............................................................................................................................ 34 
6.8 
Masking ................................................................................................................................... 35 
7 
Statistical Analysis Plan ................................................................................................................... 35 
7.1 
Outcomes ................................................................................................................................ 35 
Table 3: Study Outcomes ............................................................................................................................ 35 
7.2 
Sample Size and Power ........................................................................................................... 38 
Figure 4: Power analysis for Primary Cohort .............................................................................................. 40 
Figure 5: Power analysis for Historical Cohort ............................................................................................ 41 
7.3 
Analysis Plan ............................................................................................................................ 41 
8 
Qualitative Analysis Plan ................................................................................................................. 47 
9 
Data Quality Assurance ................................................................................................................... 48 
9.1 
Missing Data ............................................................................................................................ 48 
10 
Key Study Milestones .................................................................................................................. 49 
11 
Study Team Organization ............................................................................................................ 50 
12 
Engagement Plan ........................................................................................................................ 50 
Figure 6: Study Partners ................................................................................. Error! Bookmark not defined. 
12.1 
How patients and stakeholders have been involved in the selection of study outcomes ..... 51 
12.2 
Capabilities of the Research Team to Accomplish the Goals of the Proposed Research ....... 52 
13 
Ethics and Human Subjects Protections ..................................................................................... 52 
13.1 
Institutional Review Board ...................................................................................................... 52 
13.2 
Protected Health Information (PHI) and sources of data ....................................................... 52 
13.3 
Potential risks to subjects ....................................................................................................... 53 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
4  
 
13.4 
Adequacy of protection against risks ...................................................................................... 53 
13.5 
Potential Benefits of the Proposed Research to Human Subjects and Others ....................... 55 
13.6 
Importance of knowledge to be gained .................................................................................. 56 
13.7 
Inclusion and Accessibility ...................................................................................................... 56 
14 
Data Safety and Monitoring Plan ................................................................................................ 57 
15 
Publication and Data Sharing Policy ........................................................................................... 57 
15.1 
Publication and Presentation Policy ....................................................................................... 57 
15.2 
Key Journal Publication milestones ........................................................................................ 59 
15.3 
Making Research findings Publicly Available .......................................................................... 60 
15.4 
Making study results available to study participants after completing analyses ................... 61 
16 
References .................................................................................................................................. 61 
17 
Appendices .................................................................................................................................. 66 
17.1 
Appendix A: Patient data specifications ................................................................................. 67 
17.2 
Appendix B: Care quality data specifications .......................................................................... 77 
17.3 
Appendix C: Claims data specifications ................................................................................... 80 
17.4 
Appendix D: Organizational Survey......................................................................................... 86 
17.5 
Appendix E: Clinic descriptors table ........................................................................................ 93 
17.6 
Appendix F: Care coordinator survey...................................................................................... 95 
17.7 
Appendix G: Historical Cohort patient survey ...................................................................... 106 
17.8 
Appendix H: Primary cohort patient survey.......................................................................... 117 
17.9 
Appendix I: Historical cohort Phase 1 patient interviews ..................................................... 126 
17.10 
Appendix J: Primary cohort Phase 1 patient interviews ................................................... 130 
17.11 
Appendix K: Care coordinator Phase 1 interviews ............................................................ 133 
17.12 
Appendix L: Clinician and leader Phase 1 interviews ........................................................ 136 
17.13 
Appendix M: Study Team Organizational Chart ................................................................ 139 
 
 
 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
5  
 
List of Abbreviations 
 
AHRQ 
Agency for Healthcare Research and Quality 
BAA 
Business Associate Agreement 
BCBS 
Blue Cross and Blue Shield of MN 
CCQM-PC 
Care Coordination Quality Measure for Primary Care 
CESR 
Center for Evaluation and Survey Research 
CG-CAHPS 
CAHPS Clinician & Group Survey 
DAGs 
Directed Acyclic Graph 
DHS 
Minnesota Department of Human Services 
DUA 
Data Use Agreement 
EHR 
Electronic Health Record 
GLMMs 
Generalized Linear Mixed Models 
HCH 
Health Care Home 
HIPAA 
Health Information Privacy and Accountability Act 
HP 
HealthPartners 
HPI 
HealthPartners Institute 
ICSI 
Institute for Clinical Systems Improvement 
IRB 
Institutional Review Board 
MDH 
Minnesota Department of Health 
MN 
Minnesota 
MNCARES 
Minnesota Care Coordination Effectiveness Study  
MNCM 
MN Community Measurement 
PCORI 
Patient Centered Outcomes Research Institute 
PIs 
Principal Investigators 
SFTP 
Secure File Transfer Protocol 
STROBE 
Strengthening the Reporting of Observational Studies in Epidemiology 
 
 
 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
6  
 
Protocol Summary  
Title:  
Comparing Two Approaches to Care Coordination for High-Cost/High-
Need Patients in Primary Care 
Précis: 
This is an observational mixed method study of the outcomes from 
existing care coordination models among certified health care home 
primary care clinics in Minnesota. It combines existing quality and claims 
utilization data with a survey of patient needs and outcomes to compare 
models with and without a social worker among two cohorts of patients, 
one beginning coordination before and one after the disruptions caused 
by the COVID-19 pandemic. 
Study Aims and  
Outcome Measures 
Aim 1: To compare the healthcare quality, utilization, and patient-
reported outcomes for adult patients who receive care coordination 
services from clinics that use a medical model versus a medical/social 
model 
 
Aim 2: To identify the key components of the two models and quantify 
their association with the above outcomes. 
 
Aim 3: To explore how organizational, community, care process, and 
patient factors help explain differences in the models and outcomes. 
 
Exploratory Aim: To extend the comparative effectiveness analysis in 
the Historical Cohort by up to an additional 3 years and to investigate 
how the pandemic may have disrupted care, health, and wellbeing for 
these patients. 
Population 
Historical Patient Cohort: Patients starting care coordination in 
participating clinics between January 2018 and February 2019. 
 
Primary Patient Cohort: Patients starting care coordination in 
participating clinics between January 2021 and December 2021. 
Number of Sites 
397 potentially eligible clinics among 70 care systems 
Study Duration 
Estimated time from start of 
subject enrollment to 
completion of data analysis 
36 months 
 
 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
7  
 
Key Roles and Contact Information 
Principal Investigators:   
Leif Solberg, MD  
Senior Research Investigator 
HealthPartners Institute 
612-963-4850 
Leif.I.Solberg@HealthPartners.com  
 
Steven Dehmer, PhD  
Senior Research Investigator 
HealthPartners Institute 
952-967-5216 
Steven.P.Dehmer@HealthPartners.com  
 
Funder:   
Patient Centered Outcomes Research Institute (PCORI) 
Tabassum Majid, PhD 
Program Officer 
202-738-3726 
tmajid@pcori.org 
 
 
Partner Institutions: 
Minnesota Department of Health (MDH) 
David Kurtzon 
Director, Health Care Homes Program  
651-201-3642 
david.kurtzon@state.mn.us 
 
MN Community Measurement (MNCM) 
Liz Cinqueonce, MBA  
Chief Operating Officer  
612-746-4517 
cinqueonce@mncm.org 
 
Key Personnel: 
Leif Solberg, MD  
Steven Dehmer, PhD  
 
 
 
 
 
 
 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
8  
 
Study Protocol 
1 Introduction: Background and Scientific Rationale 
1.1 Background 
People with multiple chronic conditions or multi-morbidity make up over 2/3 of those over age 65, but 
also 1/4 of those younger than 65 who receive health care.1,2 Tinetti notes that such patients are also 
the “major users of health care services at all adult ages and account for more than 2/3 of health care 
spending.”2 Moreover, most chronic medical conditions are also associated with a higher frequency of 
depression, and Fortin et al. have shown that there is an inverse relationship between the number of 
medical conditions and patients’ quality of life.3,4  Sharma et al. found that among patients with the 14 
most disabling chronic conditions, a majority were more likely to be seen by a primary care physician 
than a specialist; therefore, it is particularly important that primary care clinics be well-organized to 
coordinate such patients’ care.5 Finally, Penm et al. demonstrated that patients in the United States 
experience poor coordination twice as often as those in other high-income countries.6 This situation has 
led many U.S. policy makers and care system leaders to emphasize the need for improved systems for 
care coordination, with a special focus on patients with complex needs and multi-morbidity. 
1.2 Evidence Gaps in Care Coordination for High-cost/High-need Patients 
An Agency for Healthcare Research and Quality (AHRQ)-sponsored review of 75 systematic reviews of 
care coordination concluded that coordination has resulted in health benefits for patients with heart 
failure, diabetes, severe mental illness, recent stroke, or depression.7 However, it also found insufficient 
evidence to assess the impact of individual components of care coordination on effectiveness. 
Ovretveit’s extensive systematic review found evidence that “most changes for better coordination 
improve quality and save resources,” but that “it depends on which approach is used, how well it is 
implemented, and on features of the environment in which a provider is operating.”8 The AHRQ review 
found that there were over 40 distinct and extremely heterogeneous definitions of care coordination, so 
the authors combined the key elements of those definitions in a new one that has subsequently become 
widely used in its original formulation. They defined care coordination as “the deliberate organization of 
patient care activities between two or more participants (including the patient) involved in a patient’s 
care to facilitate the appropriate delivery of health care services. Organizing care involves the 
marshalling of personnel and other resources needed to carry out all required patient care activities, and 
is often managed by the exchange of information among participants responsible for different aspects of 
care.”  
 
More recent systematic reviews and meta-analyses have not changed the conclusion that there is a 
large knowledge gap regarding not only how to best provide care coordination, but also the differential 
effect of various components of coordination and which other factors matter, including team roles and 
multiple levels (patient, care team, organization, and community).9-17 This evidence gap regarding both 
the coordination models and the factors that contribute to coordination success makes it difficult for 
care systems, clinicians, and payors to know which components to implement and for patients to 
identify what type of services are most important. Teams participating in the Triple Aim Initiative of the 
Institute for Clinical Systems Improvement (ICSI) identified the need to match the needs of individuals to 
the coordinator skill sets of nurses versus social workers, but found that most people with multiple 
needs require both.18 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
9  
 
1.3 How this study addresses evidence gaps  
This study aims to learn which of two major approaches to care coordination currently being used by 
primary care clinics in Minnesota (MN)—i.e., with or without a social worker on the team—provides the 
best outcomes as well as what implemented components of those approaches and which contextual 
factors are most important for effectiveness. By addressing these critical evidence gaps, the findings will 
have a direct benefit to patients, because it will allow clinics and care systems to develop and implement 
care coordination strategies that improve patient care quality, reduce utilization burden, and improve 
patient-centered outcomes. This information will also enhance the interest of payors in reimbursing the 
costs of care coordination, as witnessed by their support of this proposal. 
This study will also illuminate a related timely topic. Leaders of care systems, health plans, and 
government are increasingly realizing that health is often greatly affected by social needs (i.e., social 
determinants of health).19 Katz et al. have shown that medical care is less effective for people who have 
large social needs, and the American College of Physicians has published a position paper on the need to 
better integrate them into the health care system.20,21 As a secondary analysis of study aims, this project 
will assess whether addressing social needs through care coordination is effective and whether it 
reduces disparities.22-25 
 
The COVID-19 pandemic has brought widespread and broadly documented disruptions to daily life in the 
United States and across the globe. The pandemic has also greatly disrupted health care, requiring us to 
modify the originally planned timing for identifying study subjects while providing an opportunity to 
study changes in the outcomes from these care models through separate cohorts drawn before and 
after the main impact period in 2020. It also provided an opportunity to learn how the care models and 
multi-morbidity complex patients have been affected by the pandemic.  
1.4 Health Care Homes in Minnesota 
In 2008, MN legislation established a mechanism to certify primary care clinics as patient-centered 
Health Care Homes (HCH) that includes an extensive application and inspection, both before 
certification and again every three years for recertification.26,27 Five standards must be met for 
certification: access and communication, a searchable electronic registry, care planning, quality 
improvement, and care coordination, with specific requirements for each. HCH certification is 
administered and supported by the Minnesota Department of Health (MDH), a major partner in this 
study. 
 
By 2021, 60% of the 653 primary care practices in the state (plus 20 clinics serving Minnesotans in 
bordering states) had been HCH certified, bringing the total certified clinics to 415 (397, excluding 
pediatric clinics). All have adopted one of two models for care coordination: one that uses 
medical/nursing personnel as care coordinators for primarily medical needs and another that adds a 
social worker to address social needs more completely. In both cases, these coordination services are 
focused primarily on high-cost/high-needs patients, usually with multiple morbidities, and there is an 
extra clinic payment available from the state for patients with high complexity and state-covered 
insurance. This situation provides an ideal natural experiment for comparing the structure, process, and 
outcome differences between these two different approaches to care coordination.  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
10  
 
2 Study Aims and Outcomes  
2.1  Study Aims 
This protocol describes a comparative effectiveness observational study of two existing models of care 
coordination for high-cost/high-need patients among primary care clinics in MN.  Due to the COVID-19 
pandemic, there was substantial disruption in care during 2020 that required delaying the identification 
of a primary cohort for approximately one year. There is also substantial interest in understanding how 
approaches to care coordination compare before and after the emergence of the pandemic. Therefore, 
we plan to address the following specific aims separately for patient groups who experienced care 
coordination before (Historical cohort) and after (Primary Cohort) the worst of the disruptions: 
 
Aim 1.  To compare the healthcare quality, utilization, and patient-centered outcomes for high-
cost/high-need adult patients who receive care coordination services from clinics that use a 
“nursing/medical” model versus a “medical/social” model that includes a social worker on the 
care coordination team. 
Aim 2.  To identify the key components (e.g., personnel, content, dose, modality) of the two models and 
quantify their association with the above outcomes. 
Aim 3.  To explore how organizational, community, care process, and patient factors (including social 
determinants of health) help explain differences in the models and outcomes. 
 
In addition to these overall specific aims, the study plans to address secondary objectives of the 
comparative effectiveness in the Historical Cohort over up to an additional 3 years and to investigate 
how the COVID-19 pandemic may have disrupted care, health, and wellbeing for those patients. Figure 1 
illustrates the timing of the two cohorts and associated survey data collection. 
 
Figure 1: Timing of Patient Cohort and Survey Data Collection Efforts 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
11  
 
2.2 Study Comparators (SC-1) 
In its original certification process and its periodic recertification of clinics as Health Care Homes (HCH), 
MDH conducts both a written and a site visit-based evaluation to verify that every HCH clinic has 
implemented required standards or has obtained a variance if a particular subpart is not possible at that 
time. The relevant MDH standards (MN Administrative Rule #4764.0040) require that a HCH must 
address each of the following subparts related to care coordination: 
 
• 
1A Systematically screens patients to identify those who would benefit from care coordination 
• 
5A Collaboratively develops patient-centered goals, identifies resources to achieve goals, 
ensures consistency and continuity of care, including frequency of follow-up. 
• 
5B Designates one clinician for each patient and one care coordinator as the primary contact 
• 
5C Ensures that the clinician and care coordinator communicate with each other directly 
• 
5D Ensures the care coordinator has dedicated time to perform coordination responsibilities 
• 
6B Identifies and works with community-based organizations to facilitate the availability of 
appropriate resources for patients 
• 
Establishes and implements policies and procedures that guide who would most benefit from a 
care plan and develops the plan for identified individuals in a way that engages patient and 
team, incorporates risk assessment, is updated, and is provided to the patient 
 
It is from this information that we have developed this definition to distinguish the two care model 
comparators below. Final classification of clinics to one of each comparator model will be determined 
from data collected through a survey of care coordinators in participating clinics. 
2.2.1 Care coordination model comparator definitions 
Clinics meeting all four of the following criteria are classified as using a Medical/Social model of care 
coordination:  
 
1.  At least one social worker (licensed or not) is part of the care team at the clinic, and 
2.  The social worker is responsible for assessing and coordinating social services for care 
coordination patients at the clinic, and 
3.  The social worker routinely interacts with clinicians at that clinic, and 
4.  The social worker routinely interacts with care coordinated patients. 
 
Clinics that do not satisfy every part of this definition are classified as using a Medical/Nursing model of 
care coordination. 
 
 
2.2.2 Measuring Fidelity and Model Components 
In order to answer the main research question in Aim 1, it is crucial to have accurate knowledge about 
the type of care model present in each study clinic at the time the Primary Cohort began care 
coordination. It is also important to know how that model has changed since 2018 when the Historical 
Cohort was beginning to receive care coordination, during the pandemic in 2020, and whether any 
changes are planned during the post-coordination year for the Primary Cohort. Therefore, we will 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
12  
 
employ the three strategies in Table 1 to ascertain and confirm both the overall model used, and the 
core functions/components of care coordination delivery at each clinic throughout this time. These data 
will allow us to characterize both fidelity to the overall model and the specific ways in which there is 
variation within and between both models at the level of individual clinics. Timing of the patient cohorts 
and data collection efforts is illustrated in Figure 1. 
Table 1: Strategies for Measuring and Accounting for Fidelity to Each Care Coordination Model 
Strategy Description 
Sample Size 
Expected 
Response Rate 
Timeframe 
1 
Care Coordinator Survey 
~300-400 
100% (required) 
Q1 2022 
2 
Organizational Survey 
~50-70 
100% (required) 
Q3 2021 
3 
Primary Cohort Patient Survey 
~7,000 (~20/clinic) 60% 
Q3 2022 
 
The primary strategy will involve requiring the lead care coordinator at each participating clinic to 
complete a survey that asks about each core component. This survey will be conducted in the fall of 
2021, at a time when the clinics will have renormalized as COVID-19 pressures have declined. This timing 
will align with the Primary Cohort identification period (1/21-12/21), so it will provide a very specific 
verification of implementation models during the time the Primary cohort was being enrolled and 
receiving services. The response rate for this survey will be 100% because completing the survey is a 
requirement of participation in the study, which we have communicated to clinics throughout our 
recruitment process.  We shall also ask the lead care coordinators whether any of these core 
components have changed in any significant way during the preceding 3 years (with a focus on 2018 to 
correspond with the Historical cohort) and whether they are expected to change in the next year. These 
surveys will be preceded by semi-structured interviews with about 20 care coordinators for the purpose 
of identifying every potentially important component of care coordination needed to be included in the 
survey. The study team will design the care coordination survey to measure these components, using 
existing and validated assessment tools wherever possible.  We will draw upon measurement tools and 
definitions identified or endorsed by the Agency for Healthcare Research Quality (AHRQ) and National 
Quality Forum (NQF).28-32  We will also rely on the expertise of our external consultants Kathryn 
McDonald (lead author of the 2014 update of the AHRQ Care Coordination Measures Atlas28) and Sarah 
Scholle (previously, Vice President for Research & Analysis, National Committee for Quality Assurance) 
to translate existing measures and develop supplemental measures as needed.    
 
The second strategy for measuring fidelity to each care coordination is to supplement our understanding 
of care coordination delivery at each clinic through surveys at the overall organizational level in the fall 
of 2021.  Although the care coordinator survey will be the primary source of the specifics of care 
coordination, we will ask organizational leaders a broader set of questions to better understand the 
context, intent, resource support, and overarching design of care coordination within which each clinic 
system implements its modifications and refinements. A clear and detailed understanding of these 
factors will be fundamental to our Aim 3 analysis, but where overlap in data collection exists, it will help 
to clarify aspects of our Aim 1 and Aim 2 analyses as well. These surveys will also provide insight into the 
effect of COVID on coordination policies and processes as well as any plans for changes in the 
coordination approach through the end of our study. 
 
The third strategy is a survey of care coordination patients from our primary cohort.  This will occur in 
the fall of 2022 and will be incorporated in our planned second patient survey.  The primary goal of the 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
13  
 
patient surveys is to obtain patient-reported outcome measures for our comparative analyses.  A 
secondary goal, applicable mainly to the primary cohort, will be to ask patients how they experienced 
the various components of care coordination, which we can then compare with the designated care 
coordination model.  In cases where we see a pattern of divergence in accounting for fidelity, we will 
follow-up with clinic staff to further clarify these details. In this way, patient-level interactions will 
provide additional insight into care coordination model fidelity.  Patient surveys will not be used this 
purpose for the Historical cohort due to the difficulty in patients recalling events or situations that 
occurred years ago. Instead, that survey will focus mainly on understanding COVID-19 impacts on these 
multi-morbidity patients’ health and healthcare.  
 
2.3 Study Outcomes 
This study will evaluate comparative effectiveness measured across three outcome domains. The 
primary outcomes are identified by an asterisk: 
Patient Care Quality 
 
 
Control of blood pressure, cardiovascular disease, diabetes, asthma, 
and depression and cancer screening (from standardized state-wide 
quality measures) as well as a composite measure* of overall quality 
comprised of the percentage of all care quality outcomes for which a 
patient qualifies and meets quality criteria 
Patient Healthcare 
Utilization 
Rates of urgent care, emergency department visits*, and 
hospitalizations*, as well as primary care and specialty visits, and 
substance use programs (from claims data) as primary care and 
specialty visits, and substance use programs (from claims data). We 
will also use utilization data to estimate medical costs in U.S. dollars. 
Patient-Reported Outcomes 
 
 
Health status*, satisfaction with care*, clinician, access, and 
coordination, care integration, shared decision making, medication 
and care burden, change in insurance coverage, going without care 
due to cost or COVID-related problems, out-of-pocket medical costs, 
and changes in social needs (from patient surveys) 
 
Please see Table 3 in Section 7 (Analysis Plan) for detailed outcome definitions.  
3 Study Design 
This is an observational mixed-methods comparative effectiveness study of a natural experiment in 
which clinics have chosen one of two models of care coordination for their own reasons in order to 
comply with requirements for state certification as a medical home.  We will also address the related 
important questions raised by our stakeholder partners about what specific components of coordination 
and the local context are important and for which patients and needs coordination is most helpful. We 
will combine patient survey data (post only) with claims and quality measures (pre/post) and clinic 
coordination information to evaluate study outcomes. We will also use interviews of patients and health 
care professionals to develop study instruments and to increase understanding and validation of 
outcomes. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
14  
 
3.1 Conceptual Framework  
This study is utilizing a Care Coordination 
Measurement Framework, developed by 
McDonald et al. in the 2014 update of 
the Care Coordination Measures Atlas 
for the AHRQ (Figure 2).28 This 
framework was developed in order to 
have a way to organize measures of care 
coordination.28,33 McDonald et al. also 
developed a validated patient survey 
(the Care Coordination Quality Measure 
for Primary Care or CCQM-PC) to 
measure domains used in this study 
(e.g., facilitate transitions, assess needs 
and goals, support self-management),34 
and have also linked the framework with 
a broad quality assessment approach 
created by Donabedian to categorize 
quality factors into Structure, Process, 
and Outcome,28,35,36 as well as the 
Wagner Chronic Care Model.7 We will 
utilize the Care Coordination Measurement Framework for organizing process characteristics of each 
clinic’s coordination approach. As a consultant to this project, Dr. McDonald will be an invaluable help in 
applying it.  
 
Construct definitions in the Care Coordination Measurement Framework28  
Coordination Activities  
Actions that help achieve coordination, such as assessing needs and 
goals, facilitating transitions, or support self-management goals 
Broad Approaches 
Actions aimed at improving or facilitating coordination by building 
teamwork focused on coordination or health IT-enabled support for 
coordination 
Coordination Effects 
 
The effects of care coordination on outcomes, which will be perceived 
differently depending on who is asked and thus should be measured 
among the three represented groups  
3.2 How and why the study sites were selected 
Because care coordination is a requirement for HCH certification and because the study is conducted on 
behalf of the MDH HCH program, we will include all HCH-certified clinics in MN meeting inclusion criteria 
in the study (see Section 4.2.1). Few uncertified clinics in MN have care coordination systems, and data 
from another study comparing certified and uncertified clinics in MN demonstrate similarity between 
the two groups.37  
Figure 2: Care Coordination Measurement Framework 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
15  
 
4 Study Population and Recruitment 
4.1 Setting  
The primary study population includes all adult patients who began receiving care coordination services 
during 2018 (Historical Cohort) or 2021 (Primary Cohort) and have not opted out of research from 
participating HCH-certified primary care clinics in MN and bordering areas of Wisconsin, Iowa, and the 
Dakotas that also submit data to MN Community Measurement (MNCM). Health care professionals from 
participating clinics, including clinicians, care coordinators, and clinical leaders, will also be included as 
study subjects for some data collection. 
 
We do not expect that patients selected for care coordination will be representative of all patients at 
these clinics. They are receiving those services because they have complex multi-morbidities and are 
much more likely to have both high costs and high medical and social needs. 
4.2 Inclusion Criteria 
4.2.1 Clinics 
• 
Adult primary care clinic, HCH-certified by MDH 
• 
At least 10 adult patients/clinic currently in care coordination  
• 
Agreement by clinic or medical group leader to participate and to complete the following 
study activities:   
o Attestation confirming agreement to participation requirements and compliance 
with HCH requirements in lieu of the next recertification 
o Signed modification of the data use agreement (DUA) with MNCM for data 
transmission and use 
o Completion of contact information for care system staff during the study  
o Completion of an initial organizational questionnaire describing the medical group 
and its primary care clinics and its approach to care coordination 
o Completion of a care coordination survey by the lead care coordinator or designated 
alternate for each participating clinic 
o Co-signature on a letter to their patients inviting them to complete a survey about 
their care coordination experience 
o Cooperation with arrangements for qualitative interviews with a small sample of 
care coordinators, clinicians, clinic leaders, and patients at two time points 
o Submission of specified data elements for all eligible adult (age 18+) patients 
receiving care coordination services at two time points 
4.2.2 Patients 
• 
Age 18 or older 
• 
Historical Cohort: Receiving care coordination services in a participating clinic with a care 
coordination start date between January 2018 and February 2019  
• 
Primary Cohort: Receiving care coordination services in a participating clinic with a care 
coordination start date between January 2021 and December 2021 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
16  
 
• 
Currently insured by the MN Department of Human Services (DHS), Blue Cross Blue Shield 
MN (BCBS), UCare, Medica, or HealthPartners (HP) (for utilization-based outcomes only)  
• 
Consents to participate in interview or responds to a survey (for those data collection events 
only)  
4.2.3 HCH Clinic staff 
• 
Primary care clinicians, clinic or system leader, and lead care coordinator in participating 
HCH clinics  
• 
Consent to participate in interview and/or survey  
 
4.3 Exclusion Criteria 
4.3.1 Clinics 
• 
Pediatric HCH clinics  
• 
Clinics with fewer than 10 adult care coordination patients 
• 
HCH-certified clinics with no active care coordination program (unless temporarily due to 
COVID-19) 
 
4.3.2 Patients 
• 
Cannot complete an interview in English (interviews only) 
• 
Cannot complete a survey in English, Spanish, Somali, or Hmong (for interviews only, 
reflecting most prevalent languages in MN) 
• 
On a known research exclusion list  
 
4.4 Recruitment and Retention Strategies 
4.4.1 Clinics 
Clinic recruitment will be done at the level of the health care system to which each clinic belongs, with a 
goal of enrolling as many of the state’s HCH clinics as possible and keeping them engaged over the 
course of the study. Dr. Leif Solberg will recruit large medical groups (with 10 or more clinics) and MDH 
HCH Program staff will recruit smaller groups (fewer than 10 clinics).  
 Clinic recruitment will follow these steps:  
 
1. First, MNCM and the MDH HCH Program will distribute study information and publicity to 
the health care organizations and clinics they work with about the project and its benefits 
through their usual communication channels. 
 
2. Then a letter will be emailed to a leader of each of the 70 potentially eligible independent 
health care organizations along with a study description that includes participation 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
17  
 
requirements and benefits and a promise of a follow-up phone call soon. The requirements 
are listed above in Section 4.2. Benefits include elimination of the need for the next 
recertification, possible coverage of some costs of data submission, opportunities for input 
about the study, and timely reports and feedback about study lessons and how their clinics 
compare with others. A copy of the e-mail will be sent to the contact for the HCH programs 
and to a lead administrator. 
 
3. Follow-up calls will be made to the leader receiving the letter by either Dr. Solberg or by 
staff at the MDH HCH program (depending on system size) who are familiar with those 
organizations. At the follow-up call, the leader will have the opportunity to get questions 
answered and will either confirm a commitment to participate or will establish a date for 
that determination to be made. Health care organizational leaders in MN can commit all 
their owned clinics to such projects and we have found that their clinic leaders subsequently 
cooperate well. 
 
4. Once the organizational leadership has agreed to participate, they will identify a liaison who 
will arrange for completion of an organizational survey, identify a contact for modification of 
the DUA they already have with MNCM for data submission, and sign an attestation to the 
HCH Program that they will comply with certification standards in exchange for not having 
to undergo the next recertification process. The liaison will serve as the main ongoing 
contact for subsequent study communications or data collection needs. The organizational 
leadership will also confirm the specific clinics that will participate in the study and estimate 
the total number of patients currently receiving care coordination services at those clinics. 
 
A detailed database and tracking system will provide documentation of recruitment status and 
reminders for every eligible organization in order to provide recruitment reports to study personnel and 
PCORI.  
4.4.2  Patients 
Patients will be recruited individually for qualitative interviews, but their participation in surveys will 
depend on whether they fit the criteria for inclusion in the Historical or Primary Cohorts and then 
whether they choose to respond with a completed survey. 
4.4.2.1 Patient Interview Sampling and Recruitment (Phase 1 and 2) 
Patient interviews will occur in three rounds over two phases: once each for the Historical Cohort and 
Primary Cohort (Phase 1), and a third round among Primary Cohort patient survey respondents who 
have agreed to have an additional interview to discuss study findings (Phase 2). Each interview round is 
a separate one-time event that will involve approximately 20 patients. For the Phase 1 patient 
interviews, MDH staff will work with care coordinators from 6-10 randomly selected clinics based on the 
stratification criteria below to identify and invite patients to participate in interviews: 
 
1. Clinic care coordination model (medical/social vs. nursing/medical)  
2. Clinic geography (urban or rural based on US Census Bureau RUCA standards)  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
18  
 
3. Health system size (<10 clinics or 10 or more) 
Selection and recruitment of interview candidates will follow a convenience-sampling strategy in order 
increase the likelihood of participation and of obtaining diverse input. MDH staff will work with care 
coordinators in each selected clinic to identify and recruit their care coordination patients who they 
believe would provide diverse experiences and perspectives for study interviews. MDH will ask care 
coordinators in selected clinics to ask a few patients to contact the interview administrator with the 
Center for Evaluation and Survey Research (CESR) if they are willing to be interviewed. Care coordinators 
will be provided with an information sheet to share with selected patients with information about the 
interview, contact details, and frequently asked questions.  When interested patients contact CESR, 
professional telephone interviewers will ask a few brief screening questions and conduct the interview 
at that time or schedule the interview appointment at a time that works better for the patient.  
 
For the Phase 2 patient interviews, only Primary Cohort patients responding “yes” to a survey question 
indicating interest in receiving information about study results and providing an email address will be 
eligible. CESR will initiate outreach to a subset (depending on the number indicating interest) of these 
select individuals to conduct interviews. 
Recruitment will continue until the study team reaches the desired N in each strata for each interview 
round.  
 
4.4.2.2 Patient Survey Sampling and Recruitment (Historical and Primary Cohorts) 
Patient surveys will be conducted once for each of the Historical and Primary patient cohorts. A total of 
3,000 patients in the Historical Cohort and 7,000 patients in the Primary cohort will be sent surveys, with 
an expected response rate of 60% (yielding approximately 1,800 and 4,200 responses respectively).  
 
Survey recipients will be selected randomly. Clinics contributing small numbers of patients may be 
oversampled to ensure adequate representation. MNCM will provide CESR with the name, contact 
information, and home clinic of the sampled patient list through a secure data transfer for each cohort. 
 
CESR will manage patient survey recruitment and data collection. CESR will mail the survey invitation to 
patients with a $2 non-contingent token incentive. A web survey link and personalized PIN will be sent 
with a cover letter co-signed by the patient’s home clinic as well as the study PIs. Initial non-responders 
will be called by CESR staff at various times of the day and days of the week to maximize the probability 
of successful contact and survey completion. Patients completing the survey will receive a small (e.g., $10-
$20) retail gift card as a thank you for their time. Recruitment will continue until the contact protocol 
specified in Section 5.4.4 has been exhausted. 
4.4.3 Clinic Staff 
Clinic staff (care coordinators, clinicians, and leaders) will be recruited individually for qualitative 
interview and survey data collection. Because of the pre-existing relationship between MDH and each 
clinic, MDH is a key partner in recruiting clinic staff for data collection. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
19  
 
4.4.3.1 Care Coordinator Interview Sampling and Recruitment 
Care coordinator interviews will occur in two phases, each as a separate one-time event.  For Phase 1 
interviews, MDH staff will identify and recruit approximately 20 lead care coordinators from selected 
clinics for qualitative interviews based on the same stratified clinic-based sampling criteria described in 
Section 4.4.2 above for patients. MDH staff will provide a brief description of the study and interview 
opportunity to care coordinators with instructions to contact CESR to schedule the interview. MDH will 
also provide coordinators’ contact information to CESR as needed to complete the recruitment process 
(name, phone, email and clinic information). For Phase 2 interviews, the study team will first recruit 
from the pool of first round care coordinator interview respondents and follow the process above to 
identify additional care coordinators. Recruitment will continue until the study team reaches the desired 
N in each strata for each interview round.  
4.4.3.2 Care Coordinator Survey Recruitment  
Care coordinator surveys will be collected at one time point with the lead care coordinator from each 
participating clinic. MDH will facilitate the care coordinator survey recruitment from the smaller 
organizations they recruited and HPI staff will work with the study liaisons in the large groups they 
recruited to complete this task. Once the appropriate care coordinators have been identified, MDH or 
HPI staff will e-mail the electronic survey link to each lead care coordinator to complete. Non-
responders will receive follow-up by HPI or MDH staff via email or phone as needed until survey 
completion. Recruitment will continue until all surveys are complete.  
4.4.3.3 Clinician and Leader Interview Recruitment 
Clinician and leader interviews will also occur in two phases, each as a separate one-time event. For 
Phase 1 interviews, MDH and HPI staff will use the same stratified sampling approach described in 
Section 4.4.2 for patient interview selection to identify organizations or clinics and then work with 
leaders to recruit clinicians or leaders from selected clinics for qualitative interviews. MDH staff will 
provide a brief description of the study and interview opportunity to organizational or clinic leaders who 
will then identify clinicians or leaders to be recruited for interviews. For Phase 2 interviews, the study 
team will first recruit from the pool of first round clinician or leader interview respondents and follow 
the process above to identify additional clinicians and leaders, only if needed. Recruitment will continue 
until the sample reaches the desired N in each strata for each interview round. 
4.5 Clinic Participant Withdrawal 
4.5.1 Reasons for Participant Withdrawal 
A clinic will only be considered to be participating once its organizational leaders have agreed and 
contact information have been provided for the group and each clinic. After that, any inability to provide 
the items identified under Section 4.2.1 will be grounds for the study to assume clinic participant 
withdrawal, which can be done at any time for any reason. Patients and clinic staff may withdraw from 
survey and interview participation at any time. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
20  
 
4.5.2 Handling of Participant Withdrawals 
Clinic withdrawals will be documented in a tracking database and reported to PCORI and study 
personnel in a timely way. However, if patient data has already been submitted to MNCM and we have 
other necessary contextual information, we may continue to include their data in subsequent analyses. 
Historically, only a few health care organizations per year have discontinued HCH certification, which is 
tracked by MDH, so that is unlikely to cause much loss of data. Patient and clinical personnel withdrawal 
from surveys or interviews will be tracked by CESR and individuals will not receive further contact for 
the purpose of data collection after withdrawal. 
4.5.3 Premature Termination or Suspension of Study 
Because this is an observational study without an intervention, there is no reason to anticipate 
termination or suspension related to safety or efficacy. The study may be terminated or suspended in 
the event of a serious data or privacy breach, or if the sponsor determines the study is not meeting 
contractual obligations. 
5 Data Collection Procedures  
5.1 Care Quality, Utilization, and Patient Reported Outcomes  
As described in detail in Section 7.1, this study involves three categories of outcomes: care quality, 
healthcare utilization, and patient-centered/reported outcomes. Figure 3 shows how the collection of 
these data will be managed through a multi-step process with MNCM acting as a data coordinator. 
Figure 3: Outcome Data Collection and Management 
 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
21  
 
5.1.1 Identification of Primary and Historical patient cohorts  
Care quality, utilization, and patient-reported outcomes will be collected on both the Historical and 
Primary cohort patients identified by participating clinics in their submissions directly to MNCM. Clinics 
will identify their Historical and Primary cohorts of care coordination patients and submit their name, 
date of birth, demographics, date of starting coordination, contact information, and insurance 
information through a secure data submission portal directly to MNCM. See Appendix A: Patient data 
specifications for complete specifications of data to be submitted by clinics to MNCM. 
 
Clinic data submission will occur in two rounds, one for the Historical Cohort (care coordination 
enrollment dates from January 2018-February 2019) and one for the Primary Cohort (care coordination 
enrollment dates January 2021-December 2021). MNCM will assign a unique study ID to each patient in 
these datasets, which will be used as a linkage to the quality, utilization, and patient-reported datasets 
described below while providing confidentiality. 
5.1.2 Care Quality Outcomes data collection 
Because MNCM routinely collects standardized data from participating clinics for calculating care quality 
measures and statewide public reporting, MNCM will have care quality data in their possession on 
patients in both cohorts who met criteria for inclusion in the clinical quality measures. MNCM will 
collate applicable person-level quality measures for 2017-2019 (Historical Cohort) and 2019-2021 
(Primary Cohort). Measurement data and study ID will be provided to HPI by MNCM in a final de-
identified dataset for each patient cohort. See Appendix B: Care quality data specifications for a 
complete list of quality measures to be included.  
5.1.3 Utilization Outcomes data collection  
For patients who are plan members of a participating payor, MNCM will provide the minimum necessary 
patient identifying information to each respective payor in order to facilitate collection of utilization 
outcomes data. First, payors will verify insurance coverage through a multi-step process with MNCM so 
that all patients are matched to either the correct commercial payor or to MN DHS if on a Medicaid or 
state-sponsored plan. Then, payors will collect specified utilization data from their claims databases and 
return data to MNCM after the necessary follow-up time has accrued. Utilization data and studyID will 
be provided to HPI by MNCM in a final de-identified dataset for each patient cohort. Utilization data will 
be used to estimate medical costs in U.S. dollars by applying Total Care Relative Resource ValuesTM 
(TCRRVs), 38 which are a nationally standardized set of pricing measures that have been endorsed by the 
National Quality Forum.39,40 This method involves multiplying resource use by a standardized TCRRV 
price index to derive costs. See Appendix C: Claims data specifications for complete specifications of 
data to be provided by payor partners to MNCM.  
5.1.4 Patient Reported Outcomes data collection 
MNCM will provide the study ID and minimum necessary patient identifying information to CESR in 
order to facilitate collection of patient reported outcomes in a sub-set of both cohorts as described in 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
22  
 
Section 4.4.2.2. Survey data and study ID will be provided to HPI by CESR in a final de-identified dataset 
for each patient cohort. See Section 5.4 for details on patient survey data collection.  
5.2 Organizational Survey 
The purpose of the organizational survey is to describe the organizations and clinics participating in this 
study, each organization’s approach to care coordination, and any past or future changes in care 
coordination model or characteristics resulting from the impact of COVID and other disruptions. Each 
participating organization will submit one organizational survey and one clinic-level data form.  
5.2.1 Organizational Survey content and design 
The survey and accompanying clinic-level data form include questions designed to describe each 
organization and each of its certified clinics as well as their current general approach to care 
coordination and any changes during the COVID-19 crisis as well as changes since 2018 and any planned 
changes in the next year. This survey is the source of important contextual and independent variable 
information for all three specific aims.  The content of the survey will be developed by a survey 
workgroup and is planned to include: 
 
• 
Ownership and number of primary care sites in the care system/group 
• 
Description of the number and type of clinicians and other staff at each clinic 
• 
Care coordination goals and structure, current, previous, and planned 
• 
Number of care coordinators across the organization 
• 
Financial coverage and billing policies for coordination 
• 
Impact of COVID-19 on care coordination and plans for future changes 
• 
Patient characteristics (age, sex, race/ethnicity, language, insurance types) at each clinic 
• 
Number of care coordination patients at each clinic  
• 
Anticipated barriers to data submission 
• 
Perceived barriers, facilitators, and benefits to providing care coordination 
 
The survey and accompanying form will be piloted with select respondents from the sample pool or 
proxies with similar experience to ensure performance of the instrument. The final survey instruments 
can be found in Appendix D: Organizational Survey and Appendix E: Clinic descriptors table. 
5.2.2 Organizational Survey respondent sampling and recruitment 
Each participating organization will be asked to complete a single survey and data collection form. In 
most cases, this will be completed by an administrative leader and staff at the care system level. During 
clinic recruitment, an organizational liaison is identified. This person will assist in identifying the 
appropriate responder for each the survey and form from each organization. 
5.2.3 Organizational survey implementation  
The organizational survey will be a web-based survey built in REDCap. The link to complete the survey 
will be sent via email to the designated respondent. Certain clinic-level questions soliciting data 
elements for each clinic will be structured in an accompanying data collection form such that the 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
23  
 
primary respondent can hand-off those to an analyst or someone else better positioned to complete 
that level of detail. Initial non-responders will be sent email reminders and/or telephone calls until an 
organizational survey and form are completed for each participating organization. 
5.3 Semi-Structured Individual Interviews and Group Discussions 
In order to fully understand care coordination practices and the experience and perspectives of patients, 
lead care coordinators, and clinicians/leaders, we will conduct semi-structured qualitative interviews 
with each of these groups. Early in the study, these interviews will be used to be sure that we are not 
missing any important topics in the development of the care coordinator and patient surveys. In order 
to ensure that the patient interviews and surveys capture information important to patients, patient co-
investigators will be involved in their development and analysis. Near the end of the project, instead of 
another round of individual interviews,we will conduct discussions with groups of patients, coordinators, 
clinicians, and leaders in order to learn their reactions to our preliminary findings and recommendations 
and to help us identify the findings and messages of most interest for dissemination.  
5.3.1 General semi-structured interview methods (Phase 1) 
For the early interviews across each population (Historical and Primary patient cohorts, lead care 
coordinators, clinicians, leaders) the following standardized approach will be used, except where 
specified in the more detailed sections below. 
5.3.1.1 Instrument design 
Semi-structured interview guides will be developed based on prior research, literature, and experience. 
Interview guides will begin with rapport-building, transitioning to the primary areas of interest and 
focus, and ending with cool-down questions. Structured probes will be developed for correctness, 
clarity, and completeness of participant responses, avoiding bias and using neutral comments to 
facilitate the interview process and encourage depth in participant responses. Interviews will be 
designed to take between 20 and 45 minutes.  
5.3.1.2 Pilot testing 
All interview guides will be pilot-tested with 3-5 participant interviews sampled from the target 
population or proxies (i.e. patient co-investigators) with similar experiences. Interview guides will be 
refined before implementing with the study sample.  
5.3.1.3 Interview collection process 
Trained, experienced qualitative interviewers will conduct the interviews via phone. Interviewees will be 
given a brief description of the study and purpose of the interviews, given the opportunity to ask 
questions about the interviews, and asked for verbal consent for the interview and audio-recording. 
Audio recordings will be sent to an external company for professional transcription. Consent, field notes, 
and interview completion status will be tracked for each round of interviews electronically, either in 
REDCap or another tracking system.  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
24  
 
5.3.1.4 Incentives for participation  
After completion of the interviews, Historical and Primary cohort patient interview participants will be 
mailed a $35 gift card as a thank you for their time and effort involved in their participation. Care system 
personnel will not receive any incentive, because this is part of the participant organization expectation 
and will involve very few people at any single organization.  
5.3.1.5 Historical Cohort Patient Interviews (Phase 1) 
In contrast to the Primary Cohort patient interviews described below, the Historical Cohort patient 
interviews will occur so long after they began care coordination (3-4 years) that they will not be able to 
reliably report on specific interactions and experiences from that time. Thus, these interviews will be 
designed to document instead how patients with multiple-chronic conditions were affected by the care 
and life disruptions caused by the COVID-19 pandemic, its impacts on their health, social needs, health 
care, and social services. This will provide an opportunity to identify any disparities in care or outcomes 
among those of minority race and ethnicity or those in various age, insurance type, socioeconomic 
factors, or medical condition subgroups.   
 
Interviews will ask open-ended questions about topics such as:  
o Impact of COVID on patients’ lives – work, family, financial status, social connections, other 
social determinants 
o Impact on their health (physical and mental) and health care (How have the experienced 
changes in life, care, etc. during COVID? How did they do with COVID?) 
o Did the care coordination they received make them better able to cope with COVID stressors? 
o If space, were they able to connect with pre-COVID services during COVID? What services 
from their clinic would they have liked during this time? 
o What barriers they encountered in meeting their medical, mental, social, and physical needs 
o What other services would have been important or helpful 
 
A complete interview instrument can be found in Appendix I: Historical cohort Phase 1 patient 
interviews. 
5.3.1.6 Primary Cohort Patient Interviews (Phase 1) 
Phase 1 Interviews with primary cohort patients are intended to: (1) better understand patient 
perspectives about, and experiences with, care coordination, (2) identify the services respondents have 
received and might wish to receive, (3) learn whether the study’s proposed patient reported outcomes 
are understandable and relevant, and (4) document respondent characteristics and needs.  
 
Interviews will ask open-ended questions about topics such as:  
o Their experience with care coordination 
o What particular needs they had and how well were they are met 
o Whether there are other services they would have liked to receive 
o What barriers they encountered in trying to use coordination services 
o Whether other family members or caregivers were involved or should have been 
o What outcomes from care were most important to them  
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
25  
 
A complete interview instrument can be found in Appendix J: Primary cohort Phase 1 patient 
interviews. 
5.3.1.7 Lead Care Coordinator Interviews (Phase 1)  
Phase 1 Interviews with care coordinators are intended to: (1) identify factors important in specifying 
care coordination models, (2) identify changes that have happened in care coordination following 
disruptions from COVID-19, (3) identify clinic-specific factors affecting care coordination, including 
barriers and facilitators to use of care coordination, and (4) to obtain care coordinator perspectives on 
the most important components and processes of care coordination. This information will be used to 
develop the survey of lead care coordinators (see Section 5.5) that are key to addressing study aims.  
 
Interviews will ask open-ended questions about topics such as:  
o The personnel types and workflows involved in care coordination at their clinic 
o What they think are the strategies and resources most important for successful care 
coordination  
o The types of patients enrolled and how that process is conducted 
o The information routinely collected about patients and services and how that is accessible 
o The most common services provided and by whom they are provided 
o The social services provided and how they are provided 
o How patient follow-up is conducted and monitored 
o The main barriers to their work 
o What is needed to facilitate their work  
o Retention and turnover issues for care coordinators at their clinic 
 
A complete interview instrument can be found in Appendix K: Care coordinator Phase 1 interviews. 
5.3.1.8 Clinician and Clinic Leader Interviews (Phase 1) 
Phase 1 Clinician and leader interviews are intended to: (1) elicit opinions, perspectives, and experiences 
with care coordination that should be included in the care coordinator survey and (2) identify the most 
important barriers and facilitators of effective care coordination  
 
Interviews will ask open-ended questions about topics such as:  
o Their personal experience with referring patients to care coordination and their ongoing 
role with patients receiving care coordination.   
o How they have been engaged and communicated with by the care coordinators 
o How important care coordination has been for their patients and how satisfied they have 
been with the results and the way it is being done 
o The most important features to a successful care coordination program 
o The main barriers to successful care coordination 
o How they would change the care coordination process if they could 
 
A complete interview instrument can be found in Appendix L: Clinician and leader Phase 1 interviews. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
26  
 
5.3.2 Group Discussions (Phase 2) 
Late in the study (after we have obtained at least preliminary results), we will return to our Steering 
Committee, study liaisons from participating care systems, Primary Cohort patients, care coordinators, 
and clinicians/leaders in order to present our findings and learn the following: 
o What do stakeholders think about our findings? Which ones are most interesting and 
useful? 
o What recommendations should we make for care coordination practice? 
o What will stakeholders do with this information or hope others will do with it? 
o How can we best get the information out to people who will use it? 
 
The general process used with each group of stakeholders will be similar, including: 
o Advance distribution of a simple summary of the main findings relevant to each group (brief 
and highly visual) 
o Presentation of key findings most relevant for each group 
o Discussion led by a skilled group facilitator familiar with the study 
o Presence of representatives from MDH and HPI teams to ask and answer questions 
o Recording of sessions for on-demand access and complete capture of suggestions  
o Distribution of a short survey to live session and on-demand participants as an 
asynchronous opportunity to provide reactions and recommendations for dissemination. 
 
The process used to recruit participants and the information and questions asked will vary somewhat by 
stakeholder group, as described below. 
5.3.2.1 Care Coordinator Discussion Group (Phase 2) 
Recruitment will target care coordinators who completed Phase 1 interviews, plus others from 
participating care systems as identified by the MDH Health Care Home staff. 
 
The presentation and surveys will ask open-ended questions about topics such as:  
o The findings and whether there are any unexpected results 
o Whether recommendations are practical and can be implemented in practice  
o The perceived validity and utility of the results 
o How to best disseminate findings, both broadly and to care coordinators and care systems 
o How the results may impact their own setting and practice  
5.3.2.2 Clinician and Clinic Leader Group Discussion (Phase 2) 
Recruitment will target clinicians and leaders who completed Phase 1 interviews, plus others from 
participating care systems as identified by the MDH Health Care Home staff. 
 
The presentation and surveys will ask open-ended questions to understand perceptions regarding: 
o Perspective on the study findings and whether there are any unexpected findings 
o Whether recommendations are practical and can be implemented in practice  
o How to best disseminate findings broadly and to other clinicians and clinic leaders 
o How the results will impact their practice 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
27  
 
5.3.2.3 Primary Cohort Patient Discussion Group (Phase 2) 
Recruitment will target Primary Cohort patients who responded “Yes” to the question in the patient 
survey about whether they would like to receive information about what was learned in the study.  
 
The presentation and survey will ask open-ended questions about their reactions to the findings and 
their recommendations about what might be of most interest and value to other patients like them. We 
will also ask them about how we should best disseminate that information.  
5.4 Patient Surveys (Historical and Primary Cohorts)  
Patient surveys will be conducted to provide both contextual patient information and outcome data 
central to the study goals. The patient survey of the Historical Cohort will be conducted to provide 
information related to patient experiences during the COVID-19 pandemic. Surveys will solicit patient 
characteristics not otherwise available through the electronic health record (EHR), patient experiences 
with care coordination, and patient-reported outcomes, in the case of the Primary Cohort.  
5.4.1 Patient survey instrument design  
Patient interviews will be used to inform closed-ended surveys. Both the Historical and Primary cohort 
surveys will include questions that provide validation of the coordination process at each clinic. The 
Primary Cohort will also provide the key patient-reported outcome measures as well as patient 
characteristics. The surveys will be designed using existing survey questions with known psychometric 
properties where they exist. When no existing questions match desired survey concepts, questions will 
be developed using known best-practices for question writing.41 Patient partners will fully be engaged in 
the development of the concepts, questions, and overall survey process design.  
 
The surveys will be designed using unified mode design to minimize measurement error due to mixed 
mode implementation. Prior to full implementation, the surveys will be piloted with a small population 
similar to our target population for face validity, asking participants for their feedback on survey 
acceptability, length, and understanding (see Section 5.4.3). Ambiguous or difficult items will be 
adjusted and retested. 
5.4.2 Patient survey content  
The constructs to be included in the patient surveys include: 
o Demographic characteristics (age, sex, race/ethnicity, family, education, income, insurance) 
o Caregiver availability 
o Health status 
o Social determinant needs and changes from care coordination 
o Perceived care coordination model validation (Primary Cohort) 
o Perceived care integration 
o Satisfaction with care, access, coordination 
o Medication and care burden 
o Shared decision-making 
o Out of pocket medical costs 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
28  
 
o Going without care because of costs 
o Other factors identified in interviews 
o Personal goals and attainment  
o How the pandemic affected their lives, health, and healthcare (Historical Cohort) 
o What barriers they encountered and what services they most needed (Historical Cohort) 
 
For measures related to assessing patient-reported outcomes, we will draw upon the expertise of our 
expert consultants to ensure we are using well-validated measures, such as Clinician and Group CAHPS 
Survey (CG-CAHPS) for patient satisfaction,42,43 Elwyn’s CollaboRATE and IntegRATE measures of shared 
decision-making and integration,44,45 McDonald’s CCQM-PC,34 and the Patient Perceptions of Integrated 
Care (PPIC) survey for patient assessments of integration of care.46 
 
Complete survey instruments can be found in Appendix G: Historical Cohort patient survey and 
Appendix H: Primary cohort patient survey. 
5.4.3 Patient survey pilot testing 
We will pilot test the survey with patients who participated in the interviews and indicated at that time 
a willingness to provide this additional service. As they complete the survey, there will be added 
opportunities to identify any questions that were difficult to understand or answer as well as a place to 
indicate suggestions.  As a thank you incentive, we will provide them with another $20 gift card. 
5.4.4 Patient survey collection process  
The survey will be implemented using a sequential mixed-mode design including push-to-web and 
phone follow-up to maximize response rates and minimize potential for nonresponse bias. The initial 
survey invitation will be a mailed letter to patients with $2 as a non-contingent token incentive. The 
letter will include a URL and a unique PIN inviting the patient to complete the survey online.  As 
possible, the letter will be printed on letterhead from the respective patient clinic organization and 
signed by appropriate leaders within the care group as well as the study PIs.  
 
Surveys will be translated in Spanish, Hmong and Somali by professional translators using forward and 
back translation and subsequent reconciliation. Letters will be sent in English or Spanish for those whom 
these are the primary languages spoken. Individuals identified as Hmong or Somali speakers will be sent 
an English letter with a translated language block inviting them to call in to complete the survey with a 
bilingual telephone interviewer. Other languages will also be considered for the language blocked based 
on the prevalence in the sample population. Languages other than Spanish, Somali or Hmong that 
cannot be accommodated by CESR will be conducted using a synchronous third-party language line.  
 
After approximately two weeks of sending the letter, initial survey non-responders will be called up to 
six times by CESR-trained telephone interviewers at various times of the day and days of the week to 
maximize the probability of successful contact and survey completion. Calls will be made Monday –
Thursday between the hours of 9am and 8:30pm and Friday and Saturday between the hours of 9am 
and 5:30pm. Similar follow-up processes implemented by CESR in similar populations have yielded 
response rates in the range of 60% anticipated for this survey. We anticipate that having co-signatures 
from their source of care and the closeness of these patients with their care will help facilitate these 
response rates. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
29  
 
 
There will be a firewall between CESR and the rest of the project team to ensure that Protected Health 
Information (PHI) will never be available to anyone outside of CESR (see Figure 3).  
5.4.5 Incentives for participation in patient surveys 
All sampled patients can keep the $2 non-contingent token incentive. Patients that complete a survey 
either online or over the phone will be mailed a $10- $20 retail gift card as a thank you for their time. 
 
5.5 Care Coordinator Survey  
In order to document details of the care coordination model at each clinic, including barriers and 
facilitators to implementation, a survey will be completed by a lead care coordinator from each clinic 
site. The care coordinator survey will inform all three specific aims. The goal is to identify and 
systematically document the care model as well as all potentially important components, processes, and 
adaptations for coordination that are used there as well as any organizational, patient, or external 
environmental characteristics or resources that might be important in providing the most effective 
coordination services.  
5.5.1 Care coordinator survey instrument design  
The care coordinator survey will be designed using existing survey questions with known psychometric 
properties where they exist. When no existing questions match desired survey concepts, questions will 
be developed using known best-practices for question writing.41 MDH research team members will fully 
be engaged in the development of the concepts, questions and overall survey process.  
5.5.2 Care coordinator survey content  
Survey constructs will be derived from a combination of coordinator interviews that precede it, from 
existing validated questionnaires like the CCQM-PC and pre-identified domains identified in the AHRQ 
Care Coordination Measures Atlas. The content will include topics such as: 
• 
The current care model  
• 
Changes since 2017, during the 2020 pandemic, and any changes planned in the next two years 
• 
Staffing, panel size, mode, follow-up approach 
• 
Organizational and contextual factors considered important to outcomes 
• 
Principal barriers and facilitators 
A complete survey instruments can be found in Appendix F: Care coordinator survey. 
5.5.3 Care coordinator survey pilot testing 
Prior to full implementation, the surveys will be piloted with the care coordinators who participated in 
the earlier interviews and who indicated a willingness to do this as well. Besides completing the survey, 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
30  
 
these pilot respondents will be asked for general feedback on survey acceptability, length, and 
understanding as well as a way to highlight questions that are difficult to understand or answer. 
Ambiguous or difficult items will be revised and retested. 
5.5.4 Care coordinator survey collection process 
The care coordinator survey will be built as a web survey in REDCap by CESR. A unique URL will be 
emailed to each potential responder who has been identified by the organization as most 
knowledgeable about each clinic. Multiple reminders will be sent to initial non-responders. If needed 
phone calls will be made to encourage response until a survey is completed for each clinic. 
Individualized follow-up will be done by the HPI team for the large organizations that they recruited and 
by the MDH HCH staff for the smaller organizations that they recruited, building in both cases on the 
relationships established then. 
5.5.5 Incentives for participation in care coordinator survey  
No monetary incentives will be offered to care coordinators for completing the survey.  
5.6 Independent Variables, Measured Potential Confounders  
Contextual data needed for the analyses of all three specific aims will come from the survey of Historical 
and Primary Cohort patients, the organizational survey, the survey of a lead care coordinator at each 
clinic, and health record data shared by clinics. Table 2 describes the types of independent variables that 
will be used in this study and their sources. These variables will allow us to both fully characterize the 
patients served and the care coordination model in actual use at each clinic. They will also be used to 
test both components of the models and other features of the setting that may impact our outcomes.  
 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
31  
 
Table 2: Independent Variables 
Patient characteristicsa 
Clinic characteristicsb 
Care coordination characteristicsc 
Age 
Location 
Social worker on team 
Sex 
Ownership 
Education & experience (team 
Race/ethnicity 
Size of organization (#) of clinics) 
Types of services provided 
Insurance coverage 
Staffing 
Location (team lead, in clinic or 
Preferred language 
Services on site and in 
Care coordination patient panel 
Country of origin 
Availability of specialists and 
Certification (team lead) 
Need interpreter  
Connection with inpatient and 
Modality (phone, in person, 
Education* 
Characteristics of the overall 
clinic patient panel (age, 
Proactivity/outreach 
Employment status* 
Panel size 
Tracking/monitoring 
Household income* 
Approach to care coordination 
Payment/Charges 
Major medical conditions 
Use of data and registries 
Satisfaction with  
resources/access 
Number of care coordination 
contacts 
 
Engagement of clinicians 
Social support/social 
isolation* 
 
Others TBD based on coordinator 
interviews 
Problems with housing, food, 
safety, or transportation 
concerns* 
 
 
Caregiver needs* 
 
 
a Patient variables will be collected via patient survey(*) and/or data from care systems  
b Clinic characteristics collected using clinic survey 
c Clinic characteristics collected using clinic survey 
 
6 Data Management Plan 
6.1 Data collection 
Complete data collection methods are described in Section 5. 
Multiple steps will be taken in order to ensure adequacy and completeness of data. Upon receiving 
patient lists from MNCM, payors will verify that each patient is a plan member and also de-duplicate if a 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
32  
 
patient is identified by more than one clinic. For care quality data, MNCM has standard processes and 
algorithms for matching patients and ensuring the quality of their data. For utilization data, payors will 
utilize their operational billing claims data systems which have existing systems in place to ensure 
accuracy for payment of billing claims. For patient, clinic, and coordinator surveys, each survey will be 
designed to minimize error and missing values. Answers will be reviewed for completeness and out-of-
range responses. Care coordinator, clinician, and patient interviews will be recorded and professionally 
transcribed to ensure data accuracy. 
6.2 Data organization 
Patient data submitted by clinics to MNCM will be assembled in study-specific tables of an existing 
database server. Data transmitted between MNCM and participating payors or CESR will be exchanged 
in a standard file format (e.g., a delimited text or SAS file). Data transmitted between MNCM and HPI 
will be exchanged in a standard file format (e.g., a delimited text or SAS file) with the unique key based 
on an arbitrary (de-identified) identifier for each patient. 
 
Patient, care coordinator, and organizational survey data will be assembled in survey-specific REDCap 
database tables. Survey data and label files will be exported into .csv format and/or SAS files as directed 
by the analysts for analyses.  
 
Qualitative interview data collection will be tracked in interview-specific REDCap database tables. 
Interview data will be organized by participant type and date of data collection. Individual interviews will 
be saved as written transcripts in files which can be uploaded to a variety of qualitative analysis 
software. Each interview data collection event will have its own data folder indicating the time period 
and study population. Each data folder will contain a data file that describes the interview subjects and 
key attributes (for example demographics), individual files for each transcript, and audio files (e.g., .mp3 
or .wav format). 
6.3 Data handling 
Management and version control of patient identifying information, care quality, and utilization datasets 
will be handled by MNCM. The people responsible for managing data at MNCM are the VP of 
Technology and Innovation, the Manager of Data Collection and Integrity, one Data Integration Engineer 
and one Data Quality Specialist. MNCM will provide secure, encrypted, and user role-based access for all 
data handling controls. Beginning with data encryption, all data will be encrypted using advanced 
encryption algorithms using no less than SHA-256. This will ensure that both data in transit and at rest 
will be fully secure and protected. 
 
User access controls will govern who is able to submit, download, and review collected data. This will 
happen in three parts. First, clinics that will submit data to MNCM will utilize the MNCARES Data 
Collection Portal.  Access to this portal will be governed by MNCM and only authorized users from each 
clinic will be able to submit and validate data within the portal.  Clinic authorized users will only have 
access to their own data within the portal. Only MNCM authorized users will access to clinic portal data. 
All access to the system is logged by MNCM and reviewed on a periodic schedule.  
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
33  
 
Second, all other data transmission and collection will be facilitated through MNCM’s secure file transfer 
protocol (SFTP) server. Similar to the clinic data portal, only authorized users from each participant or 
vendor will be provided role-based access to this system for submission and retrieval. All access to the 
system is logged by MNCM and reviewed on a periodic schedule.  
 
Third and finally, as MNCM begins to compile data to be delivered per project goals, MNCM will utilize a 
secure encrypted drive to perform any analysis activity. Access to this drive will be limited to authorized 
MNCM staff who are assigned to the analysis and compilation work. All access to this drive is logged by 
MNCM and reviewed on a periodic schedule.  
For the final research datasets, MNCM programmers will protect confidentiality by randomly assigning a 
project-specific studyID for each unique patient and complying with all requirements imposed by the 
Institutional Review Board (IRB) and applicable HIPAA requirements and Data Use Agreements (DUAs). 
 
The person responsible for managing survey data is the Director of Survey and Evaluation Science at 
CESR. Version control is managed by using a single REDCap project across each data collection activity. 
As such when a web survey is completed, for example, the instrument will be marked complete and 
phone pursuit would be terminated. The singular survey response collected for each individual will be 
maintained in its original form in REDCap. If logical edits are made post-data collection to reflect skip 
patterns or other errors, they will be made outside of the REDCap environment or in a secondary 
variable so as never to overwrite source data The REDCap system is built with redundancy and is backed 
up in the HP data centers to ensure against data loss. Only those individuals who are cleared to work on 
this study will have access to the project specific REDCap environments. 
 
The people responsible for managing qualitative data are the CESR evaluation associate and Director. 
Version control is managed by keeping raw transcripts in separate folders from clean, de-identified 
transcripts. Each interview should be represented by one final transcript, and any changes to the content 
(i.e., fixing typing errors) will be saved to over-ride any past version. The original raw transcript will always 
stay available for reference in order to locate changes if necessary.  Audio recordings and raw transcripts 
will be saved as back-up to the final transcripts in the event of accidental loss of data. They will be saved 
in a separate folder so that identifiers will not be accessible to those accessing the final transcripts. The 
project drive the data are saved on is backed up on the HP network regularly. Folders containing identifiers 
will be labeled with “contains PHI” and study team members will be directed to access only final versions 
of transcript files. 
 
The people responsible for handling the de-identified transferred datasets are the HPI informatics 
programmer and study biostatistician. Transferred datasets will be stored (as is) in a designated folder in 
the project directory for the duration of the study. Analytic datasets will be derived from these transferred 
data sets and stored in a separate folder. Any residual identifying information found in the transferred 
sets will not be carried through analytic datasets.  
 
Throughout the study, PHI will not be made available to study co-investigators, project managers, or 
statistician with the exception of staff at CESR who will have access to identifiable survey and interview 
data as described above.  Patient confidentiality will be protected as specified in the protocol’s 
protection of human research subjects (Section 13) and in compliance with HIPAA and other federal, 
state, and local patient privacy procedures.  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
34  
 
6.4 Data documentation 
Each data file will have an associated data dictionary that will be developed and vetted by the relevant 
study partners (i.e., the originator and users of each data file).   
 
Data dictionaries to be developed will include the following data files: clinic data identifying care 
coordination patients, care quality data from MNCM, utilization data from each participating payor, 
interview/survey data for each administered interview/survey, and the final de-identified research 
analytic files.  Where appropriate, metadata standards will also be collaboratively developed. 
6.5  Data storage and preservation 
HPI’s networked workstation computers communicate with the larger HealthPartners (HP) corporate 
network. Data systems for storing and backing up data reside both at the HP corporate headquarters 
and in a secure offsite facility. Data are backed up daily, weekly and monthly. Computer and data needs 
are supported by the larger organization’s Information Systems & Technology Department, which 
maintains all HP computer hardware, software and data, including electronic medical record, research, 
and administrative data. 
 
Interview data is preserved for recovery by saving it in several forms: audio, raw transcripts, and clean 
transcript. Audio and raw transcript files are maintained with the transcription company, and can be re-
downloaded at any time in the event of a file loss. 
6.6 Data maintenance 
Final de-identified study data, documentation, metadata, and analytic files will be maintained in a study-
specific file folder using widely used file formats (e.g., text delimited files, PDFs, Microsoft Word 
documents, and SAS/R programs) that will require minimal maintenance during the course of the study 
or in the future. We do not have infrastructure to support a data repository for qualitative data at this 
time. 
6.7 Data sharing 
The project team outside of the data analysts will view data primarily as summaries, but will have full 
access to the clean and de-identified data sources.  Other stakeholders will primarily interact with high 
level summaries of findings in manuscripts, presentations, and/or reports.  
 
Individual consent to participate in the interviews assured participants that their information will not be 
shared on an identifiable individual level. We will not make identifiable individual level interview data or 
meeting notes available to stakeholders outside our study team, consultants named in the consent, and 
the outside transcription service. Regulatory bodies named in the interview consent forms will be able 
to access primary de-identified data for examination upon request.  
 
To facilitate the conduct of future research, we will create de-identified data sets from the completed 
project in a manner consistent with human subject protection and HIPAA privacy regulations.  These 
data sets will be kept at HPI along with data dictionaries, coding manuals, and other documentation 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
35  
 
relevant to data collection or measurement issues.  These resources will be available to the funding 
agency or to other approved investigators according to requirements imposed by the governing IRB and 
legal requirements, including HIPAA and DUAs, and organizational and/or technical constraints. 
6.8 Masking  
When possible, clinic and comparator identities will be masked from the study team until the primary 
analyses are completed, but strict masking will not be possible due to the mixed methods used for data 
collection (i.e., the combination of qualitative and quantitative data) and due to data attributes that are 
known a priori (e.g., the number of clinics associated with a care system or a care coordination model).” 
7 Statistical Analysis Plan 
This observational study will use a mixed-method convergent design,47 including exploratory and 
explanatory evaluation of the comparative effectiveness of two care coordination models implemented 
by up to 397 adult primary care clinics certified as HCH by MDH. For quantitative inferential analysis of 
each aim, standard multivariable generalized linear mixed effects regression models (GLMMs) will be 
constructed for each study outcome, using SAS (v9.4) and/or R (v4.1.2) analytic software. Causal 
inference will be informed by the PICOTS framework.48 Descriptive analyses will include summaries 
(means, standard deviations, counts, proportions) of baseline characteristics of the study population 
along with tests of bivariate association (e.g., Fisher’s exact test, Kruskal-Wallis, as appropriate) between 
patient, clinic, and contextual factors and the care coordination comparator, survey response, or study 
outcomes. Aims 2 and 3 will incorporate qualitative data obtained via patient surveys and clinic 
interviews. Qualitative findings will enhance understanding of the quantitative findings and our ability to 
address additional research questions raised by patients, partner organizations, and community 
stakeholder groups. We will follow the Strengthening the Reporting of Observational Studies in 
Epidemiology (STROBE) reporting guidelines to help ensure sufficient information in reports to allow for 
assessments of the study’s internal and external validity.49  
7.1 Outcomes 
Our outcomes fall into three categories: care quality (drawing on statewide quality measures reported 
by MNCM), utilization (drawing on insurance claims data collected by payor partners), and patient-
centered/patient-reported outcomes (collected directly from care coordination patients who agree to 
complete a survey). Section 5 describes how data will be collected. Table 3 summarizes study outcomes. 
Timing of outcome assessment will be anchored by the first date of receiving care coordination services, 
thereby creating a consistency across comparators and mitigating potential sources of bias due timing. 
Modifications to these definitions and planned analysis may arise if certain data quality and availability 
thresholds are not met.  
Table 3: Study Outcomes 
Outcome measure 
Definition 
Source 
Primary/ 
Secondary 
Follow-up 
duration 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
36  
 
Care Quality from the healthcare professional perspective 
Overall care quality 
Composite measure of overall 
quality comprised of the 
percentage of all care quality 
outcomes in which a patient 
qualifies and meets quality 
criteria for 
MNCM quality 
measures 
Primary 
12m 
Asthma care at goal* 
Asthma pts. with asthma control 
test (ACT) score >19  
MNCM quality 
measures 
Secondary 
12m 
Breast Cancer 
Screening* (up-to-
date) 
Women 50-74 yrs old who 
received a mammogram in the 
past one year 
Health plan 
claims 
Secondary 
12m 
Chlamydia Screening* 
(up-to-date) 
Female patients 16-24 yrs old 
who had a screening test for 
chlamydia 
Health plan 
claims 
Secondary 
12m 
Colorectal Cancer 
Screening* (up-to-
date) 
50-75 yr old pts. up-to-date for 
an approved screening test 
MNCM quality 
measures 
Secondary 
12m 
Depression 
improvement* 
PHQ-9 score <5 for test nearest 
to end of follow-up period    
MNCM quality 
measures 
Secondary 
12m 
Diabetes care at goal 
(including component 
measures) 
All-in-one measure of control 
A1c, blood pressure, statin use, 
& smoking 
MNCM quality 
measures 
Secondary 
12m 
Vascular care at goal 
(including component 
measures) 
All-in-one measure of control of 
blood pressure, statin use, & 
smoking + aspirin use in patients 
with vascular disease 
MNCM quality 
measures 
Secondary 
12m 
A1c control* 
Hemoglobin A1c ≤ 7% in 
patients with diabetes 
MNCM quality 
measures, 
clinic data 
Secondary 
12m 
A1c level 
Hemoglobin A1c in patients with 
diabetes 
MNCM quality 
measures, 
clinic data 
Secondary 
 
12m 
Aspirin or anti-platelet 
use* 
Aspirin use in patients with 
vascular disease unless with 
contraindication or exception 
MNCM quality 
measures 
Secondary 
12m 
Blood pressure 
control* 
< 140/90 mm Hg  
MNCM quality 
measures, 
clinic data 
Secondary 
12m 
Blood pressure level 
Systolic and diastolic blood 
pressure (mm Hg) 
MNCM quality 
measures, 
clinic data 
Secondary 
12m 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
37  
 
Body mass index 
kg/m2 (Primary Cohort only) 
MNCM quality 
measures, 
clinic data 
Secondary 
12m 
Low-density 
lipoprotein level 
mg/dL in patients with diabetes 
or vascular disease 
MNCM quality 
measures 
Secondary 
12m 
Statin use* 
Current statin use in patients 
with vascular care unless with 
contraindication or exception 
MNCM quality 
measures 
Secondary 
12m 
Tobacco use* 
Current tobacco use (tobacco 
includes any number of 
cigarettes, cigars, pipes, or 
smokeless tobacco) in patients 
with diabetes or vascular 
disease 
MNCM quality 
measures 
Secondary 
12m 
Utilization from the healthcare system perspective 
Emergency dept. visits 
# of encounters with CPT-4 E&M 
codes (99281-99288) at 
emergency dept. site 
Health plan 
claims 
Primary 
12m 
Hospitalizations 
# of hospital inpatient 
admissions ≥ 1 days 
Health plan 
claims 
Primary 
12m 
Hospital readmissions 
<30 days 
# of hospital inpatient 
admissions ≥ 1 days following a 
prior hospitalization <30 days 
Health plan 
claims 
Secondary 
12m 
Primary care visits 
# of encounters with CPT-4 E&M 
codes (99201-99215, 99381-
99429) at primary care site 
Health plan 
claims 
Secondary 
12m 
Specialty care visits 
# of encounters with CPT-4 E&M 
codes (99201-99215, 99381-
99429, 99241-99245, 92920-
93895) at primary care site 
Health plan 
claims 
Secondary 
12m 
Urgent care visits 
# of encounters with CPT-4 E&M 
codes (99201-99215, 99381-
99429) at urgent care site 
Health plan 
claims 
Secondary 
12m 
Substance use 
treatment 
Substance use treatment 
indicated by HCPCS codes 
(H0005-H0029, H0047, H2034-
H2036) 
Health plan 
claims 
Secondary 
12m 
# of chronic 
medications 
# of distinct concurrent 
dispensed medications, 
combined across drug classes 
used for chronic conditions (e.g., 
hypertension, hyperlipidemia, 
diabetes, asthma, depression)  
Health plan 
claims 
Secondary 
12m 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
38  
 
Cost of medical care 
Resource use x price index 
Health plan 
claims 
Secondary 
12m 
Patient reports from the patient/family perspective 
General health status 
NHIS 
Patient survey 
Primary 
6m-18m 
Rating of primary 
care clinic 
CG-CAHPS 
Patient survey 
Primary 
6m-18m 
Access to care 
CG-CAHPS 
Patient survey 
Secondary 
6m-18m 
Rating of care 
coordinator 
CG-CAHPS (adapted) 
Patient survey 
Secondary 
6m-18m 
Shared decision 
making  
3-item CollaboRATE44 
Patient survey   
Secondary 
6m-18m 
Perceived care 
integration  
4-item IntegRATE45 
Patient survey 
Secondary 
6m-18m 
Going without care 
due to cost 
National Health Interview 
Survey 
Patient survey 
Secondary 
6m-18m 
Out-of-pocket 
medical costs 
Medical Expenditure Panel 
Survey 
Patient survey 
Secondary 
6m-18m 
Medication and care 
burden 
Modified subset of 7-item 
Treatment Burden 
Questionnaire (TBQ)50 
Patient survey   
Secondary 
6m-18m 
Social needs  
Modified subset of 10-item CMS 
HRSN Screening Tool51 
Patient survey 
Secondary 
6m-18m 
Insurance coverage 
State Health Access Data 
Assistance Center (SHADAC) 
Coordinated State Coverage 
Survey 
Patient survey 
Secondary 
6m-18m 
* Subcomponent of overall care quality composite outcome 
Notes: MNCM = MN Community Measurement; m = months. 
7.2 Sample Size and Power 
7.2.1 Primary Cohort Sample Size and Power 
As described in Section 4, we will recruit about 397 clinics—55% with a nursing/medical and 45% with 
medical/social care coordination model—to participate in this study.  Although we recognize that we 
may not fully reach it, our goal is to recruit 100% of these clinics to participate in our study.  All of these 
clinics will be eligible to contribute to our Primary Cohort analysis.  Exact numbers of eligible patients 
from each clinic remains unknown, but we estimate that over a 12-month accrual period (01/01/2021 to 
12/31/2021), each clinic will be able to contribute 57 patients on average.  Among these patients, we 
estimate about 45%, or 26 patients per clinic, will be covered by one of our participating health 
insurance plans that will be contributing utilization outcomes.  In addition, we have budgeted to survey 
7,000 patients with an expected response rate of at least 60%, which would correspond with about 10 
or 14 patients per clinic with 100% or 75% clinic recruitment, respectively. 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
39  
 
Figure 4 illustrates the differences in outcomes we will be powered to detect under a range of clinic 
participation rates (i.e., 75%, 90%, and 100%) and number of eligible patients per clinic (ranging from 
10-50). Power and sample size analyses were conducted using PASS (v.19) software, assuming two-sided 
tests (α=0.05) with 80% power and an intra-cluster correlation of 0.01. For the composite care quality 
score, all scenarios indicate sufficient power to detect a difference in the percentage of quality 
measures at goal (range 0-100%) of at least 1%. For the primary utilization outcome, most scenarios 
indicate power to detect at least a 10% difference in the number of hospitalizations and inpatient visits. 
For the primary patient-reported outcome, all scenarios indicate the ability to detect at least a 1 point 
difference in the PROMIS-10 Global Health score (mean=50, standard deviation=10). Thus, we feel 
confident in our plan with this adaptation to achieve sufficient power to discover meaningful differences 
between care coordination models across all of our primary outcome measures for the Primary Cohort. 
 
 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
40  
 
Figure 4: Power analysis for Primary Cohort 
 
 
 
7.2.2 Historical Cohort Sample Size and Power 
Among the 397 clinics we are recruiting for this study, 329 should be eligible to contribute patients to 
our Historical Cohort analysis. Exact numbers of eligible patients from clinic remains unknown, but we 
estimate that over a 14-month accrual period (01/01/2018 to 02/28/2019), each clinic will be able to 
contribute 66 patients on average. Among these patients, we estimate about 45%, or 30 patients per 
clinic, will be covered by one of our participating health insurance plans that will be contributing 
utilization outcomes. 
 
Figure 5 illustrates the differences in outcomes we will be powered to detect under various scenarios for 
the Historical Cohort. Power and sample size analyses were conducted using PASS (v.19) software, 
assuming two-sided tests (α=0.05) with 80% power and an intra-cluster correlation of 0.01. For the 
composite care quality score, all scenarios indicate sufficient power to detect a difference in the 
percentage of quality measures at goal (range 0-100%) of at least 1%. For the primary utilization 
outcome, most scenarios indicate power to detect at least a 10% difference in the number of 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
41  
 
hospitalizations and inpatient visits. Thus, we feel confident in our plan with this adaptation to achieve 
sufficient power to discover meaningful differences between care coordination models across all of our 
primary outcome measures for the Historical Cohort. 
Figure 5: Power analysis for Historical Cohort 
 
7.3 Analysis Plan 
7.3.1 Aim 1 Analysis 
To evaluate the impact of coordination type (medical/social vs. nursing/medical) on study outcomes, we 
will specify a series of generalized linear mixed models (GLMMs). Individual patients (nested within clinic 
and organization) are the primary unit of analysis. To account for patient clustering, we will incorporate 
a random intercept for clinic into the model. Covariates will be specific to a given model (outcome) and 
will be selected based on a combination of substantive knowledge, empirical evidence, and model fit 
statistics (e.g., Bayesian Information Criterion). Investigators will first select a set of covariates that must 
be included in each model. From this initial set, a forward stepwise selection algorithm will select 
additional covariates to include until a stopping criterion is met, and the final model specification for 
each outcome is identified. 52  Random effects will be evaluated and retained in a given model based on 
model fit statistics and Wald tests for covariance parameters. Patient care quality outcomes for Aim 1 
analyses will be modeled as binary dependent variables (e.g., meeting goal vs. not meeting goal) within 
subgroups based on a patient’s status at the time of care coordination initiation (e.g. meeting criteria at 
the start of care coordination vs. not), and corresponding GLMMs will have the general form: 
 
(1)  𝜂𝜂𝑖𝑖𝑖𝑖𝑖𝑖= 𝛾𝛾000 + 𝑢𝑢0𝑗𝑗0 + 𝛾𝛾100𝑋𝑋𝑖𝑖𝑖𝑖𝑖𝑖+ 𝛾𝛾010𝐶𝐶𝐶𝐶𝑗𝑗𝑗𝑗 + 𝛾𝛾020𝑊𝑊𝑗𝑗𝑗𝑗+ 𝛾𝛾001𝑍𝑍𝑘𝑘+ 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 
 
where 𝜂𝜂 represents the log-odds of a care quality outcome (e.g., BP control); 𝛾𝛾000 is the intercept (value 
for a ‘typical’ patient at a ‘typical’ clinic); 𝑢𝑢0𝑗𝑗0 is the random intercept at the clinic level; 𝑋𝑋𝑖𝑖𝑖𝑖𝑖𝑖 is a vector 
of covariates ( age, etc.) for patient i within clinic j and organization k; 𝐶𝐶𝐶𝐶𝑗𝑗𝑗𝑗 represents the exposure 
variable of interest, care coordination type, designated at the clinic level; 𝑊𝑊𝑗𝑗𝑗𝑗 is a vector of clinic-level 
covariates; and Zk is a vector of organization-level covariates. Effect estimates will be calculated via γ 
coefficients (specifically γ010 for the relative log-odds comparing the medical/social vs. nursing/medical 
care coordination model); 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 is an error term. In addition to analyses of individual care quality 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
42  
 
outcomes, we will also evaluate a composite outcome representing the proportion of applicable care 
quality outcomes that are met by a given patient.  We intend to primarily evaluate this proportion as a 
continuous variable in a linear or tobit regression model that follows the structure in equation (1) above 
with the inclusion of a patient’s baseline composite outcome as a covariate.  However, based on the 
empirical distribution, we may also evaluate transformations to binary or multilevel categorical versions 
of this outcome as warranted.  
 
Utilization outcomes for Aim 1 analyses will be evaluated both as binomial (any occurrence vs. none) 
and as counts. GLMMs will follow the framework of equation (1) above, however, for the underlying 
distribution of count outcomes we will assume a Poisson (or negative binomial, as appropriate) 
distribution with a log (as opposed to logit) link function.  We anticipate a similar modeling approach for 
quantitative analyses of the impact of care coordination type on patient-reported outcomes (again 
adapting assumptions of the underlying distribution and corresponding link function, specific to the 
nature of each outcome variable). The main exception will be that we will not have baseline values for 
patient-reported measures, given that they will only be collected at a single time point in follow-up. 
Reporting of medical costs as a secondary outcome will follow PCORI standards.53 
 
Primary results for Aim 1 analyses will be reported as odds ratios, rate ratios, or (beta) differences 
(comparing the medical/social care coordination model to the nursing/medical model), with 95% 
confidence intervals, for each study outcome.   
 
All Aim 1 analyses as described above will be replicated for both the Primary and Historical cohorts, and 
reported separately.  To evaluate whether cohort-specific estimates statistically differ with respect to a 
given outcome, we will estimate pooled (Primary + Historical) models and evaluate the significance of 
the Wald test for the interaction term (Cohort*CC Type). Additionally, for the Historical cohort, care 
quality and utilization outcomes may potentially be available for up to 3 years from the onset of care 
coordination. Where data is available, we will estimate the impact of care coordination type on 1-year, 
2-year, and 3-year outcomes respectively, using the framework described above, within the Historical 
cohort. 
7.3.2 Aim 2 Analysis 
To identify the key components of care coordination that associate with study outcomes, we will use the 
general approach described for Aim 1 analyses, with minor adaptations. For the designated care 
coordination characteristics (Section 2.2), we will first tabulate descriptive summaries as described 
above. We will then evaluate a series of GLMMs to evaluate care coordination components (part of the 
vector of clinic-level independent variables Wjk) for each study outcome: 1) using the full model 
specification in Equation 1; and 2) stratified models restricted to the populations within care 
coordination type. These analyses will be replicated in each of the Primary and Historical cohorts as 
described in Aim 1 above. Taken together, we anticipate these sets of results will provide 
comprehensive evidence of the impact of individual components of care coordination on study 
outcomes. We may consider latent class analysis as a secondary means to classify differing approaches 
to care coordination for use in Aim 3 analysis. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
43  
 
7.3.3 Aim 3 Analysis 
Identification of significant organization, care process, and patient factors will be embedded in analyses 
for Aims 1 and 2 and will be represented as covariates in final selected models for study outcomes. 
Whereas Aim 1 analyses seek to quantify effect estimates for 𝛾𝛾010𝐶𝐶𝐶𝐶𝑗𝑗𝑗𝑗 in equation 1 above, for Aim 3 
we intend to quantify γ coefficients for patient-level factors X, clinic-level factors W, and organization-
level factors Z.  As described above, we will iteratively optimize models using a forward stepwise 
selection algorithm. Summaries for unadjusted and adjusted comparisons for all variables under study 
will be tabulated and/or plotted accordingly. Again, Aim 3 analyses will be conducted separately for the 
Primary and Historical cohorts as described above. 
7.3.4 Comparison between Patient Cohorts  
In comparing the two cohorts with respect to Aim 1, we will test and measure whether the comparative 
effectiveness of the two care coordination models differs between the two cohorts (i.e., before and 
after COVID). For example, we might learn that the two care models were performing similarly prior to 
COVID, but since COVID, patients receiving attention from a social worker in the medical/social model 
have fared comparatively better in one or more ways. With respect to Aim 2, we will test and measure 
whether the core components of the comparator care models —and other factors including size of 
patient panels for coordination, resources available to the coordinators, duration and frequency of 
encounters, mode of encounter, etc.—have shifted over time or whether the individual effects of these 
components or factors on outcomes differ between cohorts. For example, we may learn that clinics 
primarily delivering care coordination in-person corresponded with better outcomes prior to COVID, but 
after COVID, we may learn that telemedicine is being used at greater prevalence and with similar effect 
as in-person care coordination services. Finally, with respect to Aim 3, we will test and measure whether 
organizational, community, care process, and patient factors differ between cohorts and whether any 
shifts in these contextual factors have had any differential effect on patient outcomes. For example, we 
might learn that patients are experiencing challenges with non-medical factors (i.e., “social 
determinants” such as housing, employment, transportation, or insurance) at greater rates now 
compared to before the pandemic and further that the medical/social model now has a more significant 
comparative effect on these patients’ outcomes, potentially as a result. Importantly, Aims 2 and 3 will 
help us to understand what is driving any differences in the comparative effectiveness between the two 
care coordination models between the two population cohorts, should they be found to exist. 
 
To help us understand if differences we see in the Historical vs. Primary cohorts are statistically 
significant, we will run the same statistical model we used to calculate the association between care 
coordination model and a given study outcome (e.g., care quality measure) in the separate cohorts on a 
combined data set (including patients from both the Historical and Primary cohorts), with the addition 
of an interaction parameter between the patient cohort and care coordination model.  A statistical 
(Wald) test of this interaction term in the regression will help us determine if the relationship between 
choice of care coordination model and a given study outcome truly differs by cohort (timing relative to 
COVID).   
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
44  
 
Should we find no meaningful differences in the comparative effectiveness of the care coordination 
models between the two patient cohorts, our study analyses will still be enhanced by having the 
additional patient data added by the Historical cohort.  Specifically, this will allow us to conduct further 
analyses using the combined data, which will provide additional statistical power to detect meaningful 
differences in subgroup analyses or in quantifying the contributions of specific core components or 
contextual factors related to care coordination, which could help deepen our understanding of our Aim 
2 and Aim 3 findings, in particular.    
7.3.5 Summary of outcomes by clinic or organization  
In addition to the comparative effectiveness analyses described in Sections 7.3.1-7.3.4, we will 
aggregate outcomes at the clinic or organizational levels using predicted margins from the appropriate 
analytic models described above in order to provide additional information and context to clinics or 
organizations to support implementation of study findings and comparisons to their peers. 
7.3.6 Causal Inference 
7.3.6.1 Causal model underlying the research question 
The primary causal research question(s) of interest in this study is, “What is the difference in 
risk/probability of outcome Y for care coordination patients under the medical/social model, versus the 
risk/probability of outcome Y for care coordination patients under the nursing/medical model,” where Y 
refers to each of the designated study outcomes (quality measures, utilization, patient-reported 
measures).  Analyses will align with the following representation of the PICOTS framework: (1) the study 
population will consist of care coordination patients in participating Minnesota primary care clinics 
meeting criteria for the Historical or Primary Cohorts (as described in Section 4.2.2); (2) the study 
intervention is delivery of care coordination services to high-cost/high-need patients in primary care; (3) 
primary comparators will be the medical/social model versus the nursing/medical model for delivering 
care coordination, which have been implemented at the clinic level; (4) several care quality measures 
(e.g., proportion of patients with diabetes under control), healthcare utilization measures (e.g., 
hospitalizations), and patient-reported measures (e.g., health status) will be independently evaluated as 
study outcomes; (5) timing of the source data for health care quality and utilization measures will be 
based on the 12 months preceding and 12 (up to 36 for Historical cohort) months following each 
patient’s initiation of care coordination services (i.e., to account for the pre-to-post change in outcomes 
at the individual level upon exposure to the intervention) and patient-reported outcomes will be 
obtained by survey 6-18 months after initiation of care coordination services; and (6) the setting will be 
care coordination programs in Minnesota area clinics, with study data being drawn from electronic 
health records (as collected and combined from clinics statewide by MN Community Measurement), 
insurance claims, and patient surveys. 
 
Directed acyclic graphs (DAGs) are the epidemiologist’s primary tool for specifying an underlying causal 
model, its key assumptions, and potential sources of bias.54-56  Briefly, DAGs are visual representations 
consisting of nodes (variables or vectors) and arrows (causal relationships, or more appropriately, lack of 
evidence that two variables are not causally related).  They are ‘directed’ in that they depict unilateral 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
45  
 
temporal sequences (typically left to right), and ‘acyclic’ in that circular pathways cannot be included 
(variables cannot cause themselves).  We envision that our proposed quality measure and utilization 
outcome models could be represented by the following DAG, where A is the care coordination model a 
patient receives (medical/social or nursing/medical), Y is a specific study outcome (quality measure or 
utilization measure), Z is the clinic-level decision to implement one care coordination model over the 
other, Lc and Lp are vectors of measured prognostic factors at the clinic and patient level, respectively, 
and Uc and Up are vectors of unknown or unmeasured factors at the clinic patient levels, respectively.  
 
 
 
We are interested in quantifying A → Y.  Estimates for this relationship can be biased  when there are 
alternative ‘backdoor’ paths from A to Y.  In this case, we can identify multiple backdoor paths via LC, LP, 
UC, or UP.  By conditioning on measured covariates (e.g., including them in multivariable models), we can 
block backdoor pathways through LC and LP.  Assuming the DAG is correct, we are then susceptible to 
bias only through unmeasured confounders UC or UP (to the extent they exist).  To address this 
possibility, we will attempt to identify any relevant confounders that are undocumented in the data, and 
calculate e-values to determine if said confounder(s) could have plausibly accounted for the observed 
association.57  For analyses of patient reported outcomes (via survey), we modified the previous DAG as 
follows: 
 
 
 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
46  
 
We now have a selection step (S=1) between A and Y, representing completion of the survey.  All of the 
conditions of the previous DAG described above still apply – however, we now also have collider 
stratification (selection) bias, if survey responders differ from non-responders with respect to measured 
(LP) or unmeasured (UP) patient factors.  To address this, we will use inverse probability (of survey 
response) weighting, which would effectively remove the arrow from LP into S=1.  We would still be 
potentially susceptible to bias if there is strong selection by factors in UP; we will conduct additional 
sensitivity/bias analyses to place reasonable bounds on our estimates under these conditions, and 
acknowledge any limitations that remain. 
7.3.6.2 Population used to generate effect estimates 
As described in Section 4.2.2, patients will be eligible for analyses if they have been designated for care 
coordination at a participating clinic and have not opted out of research.  For utilization outcomes, 
inclusion will additionally require health insurance coverage from a participating payor.  For patient-
reported outcomes, inclusion will additionally require completion of the study survey.  As described 
above, patients will be followed for health care quality and utilization outcomes over the 12 months 
preceding and the 12-36 months following the first observed enrollment in care coordination, and 
patient-reported outcomes will be collected by survey in the 6-18 months following first enrollment in 
care coordination (Primary Cohort only).  For a discussion of analytic implications of selection by survey 
response, please see Section 7.3.6.1 above.  Differences in survey respondents vs. non-respondents will 
be assessed for potential selection bias and potential impact on validity of results. 
7.3.6.3 Timing of the outcome assessment relative to the initiation and duration of exposure 
As described above, timing in the assessment of health care quality and utilization outcomes will 
account for outcomes in the 12 months preceding initiation of care coordination in comparison to 
outcomes in the 12 months following initiation of care coordination.  Assessment of patient-reported 
outcomes will be collected in the 6-18 months following initiation of care coordination (Primary Cohort 
only). Patients’ exposure time will be the observable duration of time spent in a care coordination 
program; they will be considered exposed for the entire follow-up period for each outcome.  Exposure 
to the intervention (care coordination) will be determined at the patient level, whereas comparator 
status (choice of care coordination model) will be implemented by each clinic at the clinic level.  Due to 
these conditions, sequential temporality will be ensured, and immortal time bias will not be applicable.  
7.3.6.4 Potential confounders  
For primary analyses, covariates will be measured at or prior to baseline (entry into care coordination).  
Patient-specific follow-up duration (i.e., 6-18 months) for patient-reported outcomes collected by survey 
will be accounted for as a covariate. Section 7.3.6.1 also describes how potential confounders will be 
specified and addressed.  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
47  
 
7.3.6.5 Balance of covariates and use of propensity scores  
We do not plan to calculate propensity scores for primary analyses, as the care model decision will not 
be made by providers based on individual patient indications.  The care coordination model will be 
implemented at the clinic level, thus we anticipate patient factors will be relatively balanced between 
comparison groups.  Remaining imbalances should reflect between-clinic differences in patient 
populations and will be addressed via covariate adjustment in statistical models. 
7.3.7 Identification of participant subgroups and heterogeneity of treatment effects 
The goal of analyses to assess the heterogeneity of treatment effects is to understand whether 
comparative effectiveness between the care coordination models varies by subgroup. There is sufficient 
evidence to hypothesize that the social/medical model may be more effective for patients with high 
social needs or low socioeconomic status.58,59 We will also assess heterogeneity of treatment effects by 
age, sex, race/ethnicity, and disease status subgroups, but without an a priori hypothesis about the 
potential differences. Heterogeneity of treatment effects will be appropriately assessed using 
interaction terms in the statistical models described above.60 
7.3.8 Bias Mitigation 
To mitigate bias, we will: 1) rely on subject area experts on our project team and in the clinics to identify 
anticipated confounders for the relationship(s) between coordinated care model and study outcomes; 2) 
use directed acyclic graph methods to diagnose potential sources of confounding or selection bias, and 
identify analytic approaches to correct such biases61; 3) evaluate imbalances in empirical distributions of 
covariates with respect to care coordination model, and their associations with study outcomes 
(requisite criteria for presence of confounding); 4) employ appropriate analytic strategies (stratification, 
multivariable modeling, inverse probability weighting)62; 5) conduct sensitivity or quantitative bias 
analyses to calculate reasonable bounds for study estimates under varying assumptions57; and 6) 
acknowledge remaining limitations to be considered when interpreting our study results. 
8 Qualitative Analysis Plan 
Drs. Whitebird, JaKa, and Solberg in consultation with Dr. Crabtree will lead the qualitative data analysis 
of participant interviews. Qualitative data analysis will be approached from two perspectives: at 
baseline for survey development (Phase 1) and again in the final year (Phase 2) to understand meaning, 
implications, and recommendations regarding the study findings from the perspectives of patients, care 
coordinators, and clinicians/leaders.  
In Phase 1 patient, clinician/leader, and care coordinator interviews, we will first conduct a rapid 
analysis of any content that needs to be incorporated in the subsequent surveys of being developed for 
patients or care coordinators, which will be followed by a more deliberative directed content analysis 
approach. Following interview completion, audio recordings will be transcribed using a professional 
transcription service. These transcripts will be discussed by the analysis team as they are completed in 
order to identify immediately actionable information regarding survey development. Then, interview 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
48  
 
data (transcripts and field notes) will be analyzed using an inductive, descriptive approach to assess, 
code, and categorize the data into a priori constructs and empirical constructs arising from the interview 
data. A priori constructs will be derived from prior work and the literature that is relevant to the survey 
development focus for each participant group. Constructs will be mapped onto developing 
measurement frameworks that will form the foundation of survey development for each study 
population. Coding differences will be discussed until consensus is reached on a final coding structure 
that will then be applied to all data. The survey development team will use the findings of the data 
analysis to guide survey development for patient and care coordinator surveys.    
 
The data analysis for Phase 2 patient and care coordinator interviews will utilize a strategic and thematic 
approach to explore clinician/leader, care coordinator, and patient perceptions of study results and their 
implications.63 A thematic analysis will be applied that that is flexible and accessible in interpretation 
and application.64 Data will be systematically coded into categories, themes, and patterns emerging 
from and grounded in the data with an a priori focus on identifying the processes and elements of care 
coordination as experienced by clinicians and leaders, and the perception of study results and their 
applications from a variety of study participants. Open coding will be used to create the initial coding 
frame; data will then be coded into categories with similar characteristics.65 Classification schemes and 
typologies will be used to identify and develop emerging themes, concepts, and patterns arising from 
and grounded in the data.  
 
NVivo, a qualitative data-analysis software program, will be used for the data analysis. Issues of 
trustworthiness and rigor in the analysis defined as credibility, transferability, dependability, and 
confirmability of the data (a qualitative equivalent to validity and reliability) will be addressed through a 
number of strategies.66-68 Confirmability and dependability will be addressed by maintaining a data 
codebook and audit trail mapping decision points in the analysis. Negative case analysis will be 
conducted on individual interviews that do not fit evolving patterns in the data. Credibility will be 
addressed by having the analysis team agree on final themes and patterns in the data. Issues of 
transferability will be addressed through triangulation with other data sources. This is particularly 
important in Phase 1 as the data will be used in triangulation with other sources to inform and guide 
survey development. In Phase 2 interview data will be compared and triangulated with other study data 
to clarify and enhance quantitative findings. 
9 Data Quality Assurance  
9.1 Missing Data 
9.1.1 Methods to prevent and monitor missing data  
The absence of documentation of a care process or vital sign in the care quality database should not be 
interpreted as a missing value, but rather as indicative of a care process or test not having been 
performed. Likewise, absence of utilization indicated by billing claims almost always indicates that the 
utilization (such as a hospitalization) did not occur. Truly missing observations (e.g., SBP measured, value 
not available) will be extremely rare, undetectable, and assumed to be missing at random (perhaps 
conditional on available measures). For surveys, CESR will employ state-of-the-art methods to minimize 
unit and item nonresponse for patient-reported outcomes. For item non-response, we expect <5% 
missing data on any single item. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
49  
 
9.1.2 Statistical methods to handle missing data and statistical uncertainty due to missing data  
By definition, exposure status for Aim 1 analyses (care coordination type) will not be missing; this 
classification is determined at the level of participating clinics. Patients that are missing outcome 
measures will be excluded from analyses specific to that outcome.  For primary analyses that include 
parameters derived by interview/survey, non-responders will be excluded. In sensitivity analyses, we 
will evaluate differences in available measures by survey response status, and construct weighted (by 
likelihood of response) models to reconstruct the full population and supplement results from complete 
case analyses. Where non-survey covariate values are substantially missing (e.g., >20%), these 
covariates will not be included in primary models. Where non-survey covariate values are missing for 
<20% of a given analysis population, we will use a complete case approach for primary analyses. In 
sensitivity analyses, we will use multiple imputation (using the R package ‘MICE’) to fill in non-survey 
covariate values that are missing for <20% of a given analysis population, and reconstruct models of 
interest with partially imputed input data. Lastly, we will conduct bias analyses to quantify the potential 
impact of covariate missingness on outcome-specific model results as needed.69,70 
9.1.3 Monitoring reasons for dropout and missing data 
Utilization data may be incomplete if a patient switched insurance plans during the pre or post index 
period (note that switching among Medicaid or state-sponsored plans will generally not be lost to 
follow-up within the MN DHS data systems). The disposition of each contact attempt to complete a 
survey will be documented by the survey administrator(s). For mailings, this includes undeliverable 
addresses and active refusals. For telephone surveys, these dispositions include noncontact, refusal, 
ineligibility, and bad telephone number. 
9.1.4 Sensitivity of inferences to missing data methods and assumptions 
Given the primary and sensitivity analyses described above, we will accumulate empirical evidence on 
the potential impact of differential (informative) missingness on our results. We intend to maintain 
transparency and report all results accordingly. Interpretations based on our findings will be presented 
in the context of our assessments of missing data. 
10 Key Study Milestones   
The select dates below represent the expected data collection timeline and is subject to change. Legally 
contracted study milestones are included in the HPI contract with PCORI and are closely monitored by 
the PCORI Program Office.   
Historical Cohort Data Collection 
Expected 
completion date 
Historical Cohort data submission from clinics to MNCM  
Sep 15, 2021 
Historical Cohort patient interviews 
Aug 15, 2021 
Historical Cohort patientIDs verified with payors (multi-step process) 
Nov 30, 2021  
Historical Cohort patient surveys 
Dec 15, 2021 
Historical Cohort utilization data assembled and returned to MNCM  
Dec 31, 2021  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
50  
 
Preliminary Historical Cohort dataset assessment 
June 1, 2022 
Final Historical Cohort dataset available for analysis  
July 1, 2022 
Primary Cohort Data Collection 
Primary Cohort data submission from clinics to MNCM  
June 15, 2022 
Primary Cohort patient interviews 
April 15, 2022 
Primary Cohort patientIDs verified with payors (multi-step process) 
Aug 31, 2022 
Primary Cohort patient surveys 
Nov 15, 2022 
Primary Cohort utilization data assembled and returned to MNCM  
May 1, 2023* 
Preliminary Primary Cohort dataset assessment 
Aug 1, 2023 
Final Primary Cohort dataset available for analysis  
Nov 1, 2023 
*Time gap is for utilization data to accrue for approximately 12 months from date of CC enrollment 
Other data collection 
Organizational survey and clinic descriptors table 
Nov 1, 2021 
Care Coordinator and Clinician/Leader Interviews (Phase 1) 
Oct 1, 2021 
Care Coordinator Survey 
April 15, 2022 
Patient, Care Coordinator, and Clinician/Leader Interviews (Phase 2) 
Dec 31, 2023 
Analysis and Dissemination  
All data analyses complete for all study aims 
April 30, 2024 
Draft Final Research Report submitted to PCORI 
June 30, 2024 
 
11 Study Team Organization 
The study will be led by the co-Principal Investigators (Leif Solberg, MD and Steven Dehmer, PhD) with 
the assistance of a Principal Project Manager (Anna Bergdall, MPH). They will meet weekly along with 
additional project managers as a Core Team to plan every aspect of the study. Their plans and questions 
will be discussed in a biweekly Executive Team that includes leaders from the main partner 
organizations (MNCM and MDH) as well as a patient partner and a rotating member of the investigator 
team. Major concerns, progress, and decisions will also be reviewed at a quarterly meeting of the 
Steering Committee (which includes representatives of every collaborating organization, all patient 
partners, all consultants, and all co-investigators). Implementation of those plans will occur through 
standing and ad hoc work groups, including an operations committee for the three principal 
organizations, a survey/interview committee, an analysis committee, and a dissemination/publication 
committee. See Appendix M: Study Team Organizational Chart for a complete study team 
organizational chart.  
12 Engagement Plan 
This study is led by HealthPartners Institute in collaboration with MDH, Minnesota Community 
Measurement (MNCM), a statewide nonprofit for public reporting on standardized performance 
measures, the MN Department of Human Services (DHS), payor partners including HealthPartners, 
UCare, Blue Cross Blue Shield of MN (BCBS), and ICSI. This collaboration ensures wide dissemination of 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
51  
 
learnings throughout the state and that the findings, which will also be broadly applicable outside of 
MN, can be shared nationally by leveraging each partner’s national networks. 
 
From the beginning of the proposal planning, we have involved our major partners (MDH HCH program 
and MNCM) and patient investigators in all planning and will continue to do so through their 
participation in the Executive Committee and relevant subgroups. In order to foster awareness and 
engagement by them and all other collaborators in this large complex project, we have included 
everyone in Figure 6 below in a Steering Committee that meets quarterly and are sending them monthly 
email updates of study progress, both with opportunities to provide input. Most are also participants in 
other subgroups and committees relevant to their roles and expertise. In order to assure broad 
stakeholder input, one PI is a member of the HCH Advisory Committee that includes patients, care 
system and payor leaders and other community representatives. We are particularly interested in 
having the clinics feel engaged, so we held a kickoff webinar for them and will distribute updates 
regularly through the newsletters of the HCH program and MNCM that reach every care system 
participant. Finally, qualitative interviews with care system leaders are part of our data collection plan at 
two points in the study.  
 
12.1 How patients and stakeholders have been involved in the selection of study 
outcomes 
Patient co-investigators and organizational partners were involved in proposal development, including 
the study outcomes and in the several major revisions that have occurred since being awarded. Input 
has been obtained through meetings, individual outreach, and emailed edits to the materials. We also 
reviewed the outcome measures for this project with the multi-stakeholder Advisory Committee for the 
HCH program, which includes representatives of consumers, clinics, payers, employers, and others. We 
will continue to obtain ongoing input from patients and stakeholders on study outcomes. 
Figure 6: Study Partners 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
52  
 
12.2 Capabilities of the Research Team to Accomplish the Goals of the Proposed 
Research 
The core team at HPI are supported by key research consultants, including Dr. Whitebird, a professor 
and mixed-methods research expert who worked as a care coordinator early in her career and has an in 
depth understanding of that role and responsibility.  We also have Dr. Benjamin Crabtree, an 
internationally renowned leader of qualitative and mixed methods studies at Rutgers Medical School, 
Dr. Glyn Elwyn, senior scientist at Dartmouth who is the world leader of shared decision-making and 
developer of both the CollaboRATE survey measure of shared decision-making and the IntegRATE survey 
measure of care coordination, both from the patient’s perspective.44,45,71 He works with Dr. Eugene 
Nelson, a leading authority on patient-reported outcomes and patient engagement and a long-term 
colleague.  They are joined by a key new important consultant, Dr. Kathryn McDonald, an expert on 
coordination measures frameworks and leader in development of the AHRQ Care Coordination 
Measures Atlas as well as Dr. Sarah Hudson Scholle, director of research for the National Committee for 
Quality Assurance (NCQA).  
 
Our investigator team is rounded out and greatly enhanced by our three patient and one family member 
co-investigators who bring diverse perspectives and experiences with the medical system. They have 
participated in the development of this proposal from the beginning and will be involved in every 
project committee or workgroup and every important decision. 
 
13 Ethics and Human Subjects Protections 
13.1 Institutional Review Board 
Research activities at all study sites (HPI, MNCM, and MDH) is overseen by HealthPartners Institutional 
Review Board (IRB) FWA# 00000106, 8170 33rd Ave S, MS 23301A, Bloomington MN 55425. 
13.2 Protected Health Information (PHI) and sources of data 
All necessary data to determine patient-level study inclusion and evaluate care quality and utilization 
outcomes in the two patient cohorts are derived from EHR and claims data that will be reported by 
clinics and payors to MNCM. See Appendices A and C for detailed descriptions of the PHI provided by 
clinics and plans to MNCM for this purpose. All transfers of PHI to MNCM will be done under 
appropriate data agreements between MNCM and participating clinics and payors. MNCM will provide 
HPI with a final fully de-identified final dataset for research analysis; the HPI study team will not have 
access to the PHI used to construct the final research datasets.  
 
Minimum necessary patient identifying information on a sampled sub-set of each patient cohort 
(n=3,000 Historical Cohort, n=7,000 Primary Cohort) will be provided by MNCM to CESR to conduct 
patient surveys to determine patient-reported outcomes. PHI for this purpose includes name, contact 
information, and clinic. All qualitative interviews with patients will be done on an opt-in basis with 
informed consent and will be used to help determine survey constructs and interpret study findings. 
CESR will provide HPI with a fully de-identified dataset for each survey collection to the HPI study team.  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
53  
 
Care coordinator surveys, care coordinator interviews, and clinician/leader interviews will not involve 
PHI. Instead, these data will provide professional information about organizations, workflows, 
responsibilities, and care models. However, all data collection with care coordinators, clinicians, and 
leaders will be collected with informed consent and only the minimum necessary study team staff will 
have access to the identifiable data. HPI study team members outside of CESR staff will not have any 
access to PHI used for any of the described purposes. 
 
Detailed information about recruitment, sampling, and data collection for each of these data sources 
can be found in Sections 4 and 5.  
13.3 Potential risks to subjects 
 
This study does not involve any interventions. Potential risks to subjects relate primarily to loss of 
privacy and confidentiality. Privacy loss could occur during PHI transfers from clinics to MNCM, between 
MNCM and payor partners, or from MNCM to CESR. Privacy loss could also occur through survey and 
interview data collection. Measures to minimize these risks are discussed below. 
13.4 Adequacy of protection against risks 
13.4.1 Protection of Informed Consent 
13.4.1.1 EHR, care quality, and claims data collection 
This study will plan to operate under a waiver of documentation of informed consent for patients for the 
use of EHR-derived and claims data for the following reasons:  
1. Data exchanged for use in this study does not present more than minimal risk of harm to 
subjects and is exchanged between clinics, payors, and MNCM under legal data privacy 
agreements and under rigorous data security protocols;  
2. The principal risk of data collection in this study is breach of confidentiality;  
3. The inclusion of patients in the study dataset by clinics is based on participation in care 
coordination during specified time periods, and would be impractical to obtain written informed 
consent; and 
4. The study will provide pertinent information for informed consent for the sub-set of patients 
receiving surveys and/or participating in interviews. 
13.4.1.2 Patient Surveys  
Patient survey respondents will consent to the surveys through an affirmation demonstrated by survey 
completion. The patient survey will be mailed with a cover letter explaining elements of informed 
consent for completion of the survey, which will include pertinent information about the broader study. 
A completed survey indicates consent to use the survey data for research.  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
54  
 
13.4.1.3 Care Coordinator and Organizational Surveys  
Care coordinator surveys will be distributed with a cover letter outlining all elements of consent, but 
they will not be asked to sign a formal consent statement or form. The care coordinator survey data will 
be used primarily as a means to collect factual information about how care coordination is implemented 
specifically in each clinic. However, these employees will be assured of their privacy. 
 
Organizational surveys will not be distributed with elements of informed consent. The organizational 
survey data will be used primarily as a means to collect factual information about care models and 
organizational information across participating care systems.  
 
For both surveys, the respondents are not research subjects, but instead are leaders participating in the 
survey as a means of partnership with the study team on behalf of their clinic or organization.  
13.4.1.4 Semi-Structured Interviews 
A verbal consent process will be conducted for the collection of all patient, care coordinator, and 
clinician/leader interviews. Only respondents who verbally consent to participate will be interviewed 
and included in the dataset. All verbal consent will be outlined in interviewer scripts and documented by 
study staff in the appropriate method-specific tracking system. 
13.4.2 HIPAA protections 
HIPAA makes a special provision for a waiver of HIPAA authorization to use PHI for research under 
certain conditions (the HIPAA Privacy Rule), which this study meets.  
 
Specifically in this study, PHI will only be disclosed to MNCM under existing business associate or DUAs, 
which outline privacy regulation between MNCM, clinics, and payors in compliance with HIPAA 
regulations. Entities exchanging data with MNCM for this study will be responsible for assuring privacy 
measures are properly followed. Additionally, PHI sent by MNCM to CESR for the purpose of facilitating 
survey data collection will be done under an appropriate DUA and will only be accessible to the recipient 
of that data and CESR staff.  
 
The final research dataset provided by MNCM to HPI will be fully de-identified per Privacy Rule 
definitions and will therefore no longer require HIPAA protection. Still, an appropriate data use 
agreement will be made for receipt of the final de-identified dataset to safeguard protection of patient 
privacy. 
13.4.3 Protection of confidentiality and data security  
The study team has extensive experience in health services research and clinical research with human 
subjects, with procedures to safeguard privacy and personal information. All study records are protected 
by:  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
55  
 
1. Use of untraceable studyID numbers instead of names wherever possible 
2. Password protection as well as firewalls 
3. Strong user login authentication on all electronic devices 
4. Physical security for all electronic devices containing personal information  
5. Locked storage for all paper records in a secure location 
Data will be retained in secure storage following the completion of the study in accordance with 
Minnesota and federal law. We guard against the potential for breach of subject confidentiality through 
a multi-layered system of data protection policies, processes, staff training, software safeguards and 
physical security measures for both paper and electronic data involved in research, at both MNCM, 
CESR, and HPI. 
 
The following measures will be taken to protect subjects from the risk of breach of confidentiality at 
both MNCM, CESR, and HPI:  
1. All data collected in the study will be identified by using an arbitrary and unique studyID number 
to each patient.  
2. A file containing a link between the studyID and individually identifying information will be 
maintained at by a study team programmer at MNCM through the conclusion of the study.  
3. A cross-walk table linking the studyID to a patient identity will be destroyed within 6 months 
after the linked databases needed to complete study analyses are completed.  
4. All electronic study data will be maintained in a computerized database residing on a username- 
and password-protected file-server to which only the study team members will have access.  
5. All study-related paper documents containing individually identifiable information will be 
maintained in locked file cabinets. 
For protection of confidentiality of semi-structured interview participants, we will follow all of the above 
measures, which also apply to audio recordings and transcripts. 
 
To protect the confidentiality of any clinic or organization employee participating in a survey, we will not 
allow anyone outside of the research team to know the identity of those respondents. All of the 
protection to electronic data sources, described above, also apply to survey collection. 
 
13.5 Potential Benefits of the Proposed Research to Human Subjects and Others 
Patients in the study will have no defined personal benefit from participating in this project. 
Compensation for the time to complete surveys and interviews will be minimal but appropriate 
according to effort involved with participation. All patients receiving a survey following enrollment will 
receive a small $2 non-contingent incentive with the survey to increase response rates. Patients 
completing a survey will receive another $10 gift card for their time.  Interview participants will be 
offered compensation of a $35 gift card for their time. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
56  
 
13.6 Importance of knowledge to be gained 
To date, no study has been able to compare models of care coordination in the systematic way 
described in this protocol. If the study reveals that one care coordination model results in better 
outcomes, then MDH and care systems can focus on promoting use of that care coordination model. If 
the study does not reveal better outcomes in one care coordination model over the other, then MDH 
and care systems can focus on optimal implementation of each care system’s chosen model. Ultimately, 
the knowledge in this study will guide future implementation of care coordination and will serve as a 
resource to care systems in other settings as can use evidence about effectiveness to guide care 
coordination programs.  
13.7 Inclusion and Accessibility 
13.7.1 Inclusion of Women 
All eligible patients utilizing care coordination services at a participating clinic will be included, so 
women should be included in the same proportion as they have those criteria and receive care in the 
study clinics. Preliminary data suggests that approximately 62% of patients receiving care coordination 
services are women. It is also likely that the great majority of care coordinators and other clinic 
personnel will also be women. 
13.7.2 Inclusion of Ethnic and/or Racial Minorities 
All eligible patients utilizing care coordination services at a participating clinic will be included, so ethnic 
and racial minorities should be included in the same proportion as they have those criteria and receive 
care in the study clinics. No one will be excluded based on language spoken. Surveys will be 
administered to Spanish, Hmong and Somali-speaking patients by CESR bilingual interviewers. Patients 
that speak languages other than English and the three non-English languages most common across the 
state will be facilitated using a third-party language line. Inclusion is important because one of the 
possible outcomes from our analysis of this study’s data is an evaluation of any racial or ethnic disparity 
in participation or outcomes of care coordination.   
13.7.3 Inclusion of Children 
Children will not be included in this study, because the proportion of children with multiple chronic 
conditions or high complexity is too low to provide a sample large enough to analyze without selective 
recruiting that is incompatible with the way we have developed to obtain patient data. 
13.7.4 Other Special Populations 
The patient data will include all adults cared for in the study clinics who received care coordination 
services and have health insurance coverage from one of our participating payors, which includes the 
MN DHS that oversees coverage for all Medicaid patients in the state. Therefore, subjects will include all 
population groups in proportion to their representation in these clinics and with those 
conditions/needs.  In particular, it will include persons with complex medical conditions, high social 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
57  
 
needs, the elderly, and minorities, because those are the types of patients most likely to be referred for 
care coordination services.  
14 Data Safety and Monitoring Plan 
The Principal Investigators (PIs) are responsible for monitoring the data and assuring protocol 
compliance.  
Because this is an observational study (no intervention involved), risks to patients are minimal and 
involve the risk of violation of confidentiality of their identity (revealing that the patient utilized care 
coordination services) or of their care quality, utilization, or survey-provided information to 
organizations who already maintain or exchange such information about the study patients. Risks to 
care coordinators and other clinic personnel who will be interviewed are similarly minimal, because they 
will not be asked about any personal information, only about the way that the care coordination process 
functions in their clinic and their recommendations about ways that it might be improved.  
 
Several strategies will be employed to protect against risks to patients. First and foremost, we will 
employ the “minimum necessary” principal to only collect or use sensitive or personally identifiable 
information as necessary to conduct the study. Core identifying data will include the patient’s name, 
date of birth, and plan member ID—which are data elements routinely shared among all the data 
partners for operational purposes. By design, the study analysis team will only receive de-identified 
data. In addition, data systems implemented for exchanging data will employ technical security 
measures of the partner with the most stringent security requirements. For clinic personnel, it is less 
feasible to completely de-identify their information, because it is important for the analysis that we 
retain a linkage between their information and their clinic and position in the clinic, but we can 
anonymize their identity and avoid asking any questions about their views or personal health 
information that could put them at risk. 
 
Adverse events or other problems are not anticipated. In the unlikely event that such events occur, the 
PIs are responsible for reporting to the IRB and any appropriate funding and regulatory agencies any 
serious, unanticipated and related adverse events or unanticipated problems involving risks to subjects 
or others. The study’s funder (PCORI) and the HealthPartners’ IRB will be informed of adverse events 
within 10 working days of the event becoming known to the PIs. The PIs or the IRB have the authority to 
stop or suspend the study or require modifications.  
15 Publication and Data Sharing Policy  
15.1 Publication and Presentation Policy 
The following policies and procedures are designed to facilitate more good publications and 
presentations, fewer presentations that don’t lead to publications, and inclusion of as many project-
associated personnel as co-authors as possible.  

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
58  
 
15.1.1 Publication and Presentation Policy Goals 
1. To encourage publication of as many good papers in indexed journals as possible 
2. To limit presentations to co-existing papers or to audiences that will use the information 
3. To assure inclusion, fairness, and appropriate recognition and acknowledgements 
4. To control project analytic resource use for addressing priority project aims 
5. To prevent inappropriate, duplicate, or conflicting statements or use of data 
15.1.2 Publication and Presentation Policy Process 
1. The Executive Team will establish policies and procedures, suggest key needed articles, coordinate 
use of project resources, and resolve conflicts 
2. Anyone wishing to take the lead on writing an article (or making a presentation) that uses study data 
or concepts should submit a very brief abstract (see below) to the PI for review 
3. Any academic conference presentation should be viewed as a complement to a publication 
submission, either before or after the presentation 
4. After review, proposed abstracts will be circulated to all interested parties, so anyone who might 
want to be included or to suggest changes can do so.  Thereafter the lead author becomes the chair 
of that paper-writing group. 
5. Both the original abstract and the final draft article must be approved by the Executive Team or a PI 
before submission  
6. The first author is the one who prepares the initial draft, coordinates the input and contributions of 
co-authors, and has the last word in any differences of opinion. Co-authors are in approximate order 
relative to contribution as decided by the first author 
7. The success of MNCARES depends on many people, so we should err on the side of including anyone 
who wants to be a co-author, subject to #8 below. 
8. Requirements for being listed as a co-author are from the "Uniform Requirements for Manuscripts 
Submitted to Biomedical Journals" from the International Committee of Medical Journal Editors 
(ICMJE) and can be accessed at the following website: http://www.icmje.org/about-
icmje/faqs/icmje-recommendations/ 
All 4 of the following conditions must be met: 
1. Substantial contributions to the conception or design of the work or the acquisition, analysis, 
or interpretation of data for the work;  
2. Drafting the work or revising it critically for important intellectual content;  
3. Final approval of the version to be published;  
4. Agreement to be accountable for all aspects of the work in ensuring that questions related to 
the accuracy or integrity of any part of the work are appropriately investigated and resolved 
Participation solely in the acquisition of funding or the collection of data does not justify authorship. 
General supervision of the research group is also not sufficient for authorship. 
9. Any potential co-author who does not respond to requests for reactions to drafts or the final version 
of the paper in a timely way will be assumed to no longer wish to participate in authorship 
10. PCORI (PO, PA, CA) must be notified promptly about any accepted papers and pre-print and final 
copies should be submitted through the Publications section of the PCORI Online Portal. All papers 
must acknowledge grant support and use the following notation: 
 
"This work was supported through a Patient-Centered Outcomes Research Institute (PCORI) Project 
Program Award (IHS-2019C1-15625). All statements in this report, including its findings and 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
59  
 
conclusions, are solely those of the authors and do not necessarily represent the official views of the 
Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology 
Committee.” 
 
11. Most papers should err on the side of over-acknowledging the clinics, organizations, and people 
who contributed to the paper’s content 
15.1.3 Publication and Presentation Policy Roles and Responsibilities  
Roles and Responsibilities of Writing Team members include: 
Lead Author 
• 
Submitting/revising the proposal abstract 
• 
Identifying interested potential co-authors and making the final decision on 
which have satisfied the ICMJE requirements 
• 
Identification of a realistic submission date goal 
• 
Progress on meetings and paper drafts 
• 
Developing the first draft of the paper 
• 
Coordinating with appropriate staff to integrate data findings 
• 
Scheduling and facilitating calls/meetings, as needed 
• 
Distributing drafts to the writing team for input  
• 
Making sure the final draft is copy-edited (Ann Harste to assist) 
Co-Author 
• 
Contributing ideas and references, not just copy-editing 
• 
Responding to each draft distribution within the requested time 
Mentor 
When an investigator is the first author but has limited experience with publication, 
they should identify a mentor who can coach and/or support them in the 
development and completion of the manuscript in a timely way  
Co-PIs &  
Executive Team 
Responsible for approving proposal abstracts, prioritizing papers, identifying co-
authors, and monitoring progress. 
 
15.1.4 Abstract for Proposed Publications and Presentations 
Abstracts for proposed publications and presentations should describe the following elements: 
1. Descriptive title and date 
2. Main goal, question, or hypothesis to be addressed in the paper 
3. Brief rationale and description of analyses 
4. Data needed 
5. Lead author and tentative co-authors 
6. Audience and journal targeted 
7. Target date for submission  
15.2 Key Journal Publication milestones 
1. Proposal submission to the PIs 
2. Proposal approval 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
60  
 
3. First draft to co-authors 
4. Submission to a journal 
5. Journal decision 
6. Resubmissions 
7. Acceptance 
8. Publication  
15.3 Making Research findings Publicly Available 
15.3.1 Public Summary of Findings 
Besides our primary outcomes paper, we will also produce a summary of research findings for patients 
and the general public in order to convey our findings in a “comprehensible and useful manner to 
patients and providers to use in making health care decisions.” PCORI and our patient co-investigators 
and advisory group will help us to develop the summary and ensure that it is available in public-access 
format. 
15.3.2 Public Access to Journal Articles 
An electronic copy of the final peer-reviewed publication of our primary outcomes will be submitted to 
the National Library of Medicine’s PubMed Central to be made available publicly. Costs for this are 
provided by PCORI. 
15.3.3 Presentations and PCORI-initiated Events 
We will attend PCORI meetings or other events to present research findings as requested by PCORI. 
Expenses for these trips will be covered by PCORI. 
15.3.4 Other public and professional dissemination  
Our imperative will be to provide any useful information to the care systems, insurers, and other 
organizations that want to improve the effectiveness, efficiency, and patient-centeredness of their own 
care coordination efforts. Most of these organizations in the MN area are active collaborators in the 
proposed study, so they are very interested in using the findings. Most importantly, the HCH Program 
will certainly put the lessons to use in their certification and recertification relationships with all certified 
HCH clinics and those applying for certification in the future. Besides individual contacts and visits, this 
program also has an annual learning day event that brings together representatives from all certified 
clinics and this topic will be a major focus for these events during and after the grant. ICSI (our regional 
quality improvement organization with most payors as sponsors and most large care systems as 
members) has many ways to communicate with most of the clinics in the region (bordering states as 
well as MN), and because they are also a founding member of the Network for Regional Healthcare 
Improvement (30 member organizations nationally), they will be able to disseminate the lessons widely.  
Finally, we expect that our health system and state government collaborators will be eager to make use 
of the lessons that they have been part of producing. Hopefully, that will include modifications in the 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
61  
 
current rules for clinic certification as well as better payment for the most effective kinds of care 
coordination. 
15.4 Making study results available to study participants after completing analyses  
There are two kinds of study participants – patients and care system members. During the patient 
survey, we will ask if the respondent would like to receive a focused lay-language report of our findings 
that are most relevant to patients and send them such by email. Our patient partners will assist in 
developing this and other ways to disseminate the information to patients and the community in a way 
that is meaningful for them. For example, one of our patient partners has his own radio program. We 
have promised participating care systems that we will provide them with their results in relation to the 
average so that they can see how their clinics compare. We will also provide citations or links to 
published papers and summaries of the most pertinent lessons, especially those that would help them in 
making decisions about improving care coordination. We also plan to make use of our many 
stakeholders and advisory groups to identify those lessons of most interest. 
16 References 
1. 
Glynn LG, Valderas JM, Healy P, et al. The prevalence of multimorbidity in primary care and its 
effect on health care utilization and cost. Fam Pract. 2011;28(5):516-523. 
2. 
Tinetti ME, Fried TR, Boyd CM. Designing health care for the most common chronic condition--
multimorbidity. JAMA. 2012;307(23):2493-2494. 
3. 
Fortin M, Lapointe L, Hudon C, Vanasse A, Ntetu AL, Maltais D. Multimorbidity and quality of life 
in primary care: a systematic review. Health Qual Life Outcomes. 2004;2:51. 
4. 
Read JR, Sharpe L, Modini M, Dear BF. Multimorbidity and depression: A systematic review and 
meta-analysis. J Affect Disord. 2017;221:36-46. 
5. 
Sharma MA, Cheng N, Moore M, Coffman M, Bazemore AW. Patients with high-cost chronic 
conditions rely heavily on primary care physicians. J Am Board Fam Med. 2014;27(1):11-12. 
6. 
Penm J, MacKinnon NJ, Strakowski SM, Ying J, Doty MM. Minding the Gap: Factors Associated 
With Primary Care Coordination of Adults in 11 Countries. Ann Fam Med. 2017;15(2):113-119. 
7. 
McDonald KM, Sundaram V, Bravata DM, et al. Care Coordination. Rockville, MD: Stanford 
University-UCSF Evidence-based Practice Center; 2007. 04(07)-0051-7. 
8. 
Ovretveit J. Does clinical coordination improve quality and save money? London, 2011. 
9. 
Backhouse A, Ukoumunne OC, Richards DA, McCabe R, Watkins R, Dickens C. The effectiveness of 
community-based coordinating interventions in dementia care: a meta-analysis and subgroup 
analysis of intervention components. BMC Health Serv Res. 2017;17(1):717. 
10. 
Conway A, O'Donnell C, Yates P. The Effectiveness of the Nurse Care Coordinator Role on Patient-
Reported and Health Service Outcomes: A Systematic Review. Eval Health Prof. 
2017:163278717734610. 
11. 
Dy SM, Apostol C, Martinez KA, Aslakson RA. Continuity, coordination, and transitions of care for 
patients with serious and advanced illness: a systematic review of interventions. J Palliat Med. 
2013;16(4):436-445. 
12. 
Falconer E, Kho D, Docherty JP. Use of technology for care coordination initiatives for patients 
with mental health issues: a systematic literature review. Neuropsychiatr Dis Treat. 
2018;14:2337-2349. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
62  
 
13. 
Gorin SS, Haggstrom D, Han PKJ, Fairfield KM, Krebs P, Clauser SB. Cancer Care Coordination: a 
Systematic Review and Meta-Analysis of Over 30 Years of Empirical Studies. Ann Behav Med. 
2017;51(4):532-546. 
14. 
Kroll-Desrosiers AR, Crawford SL, Moore Simas TA, Rosen AK, Mattocks KM. Improving Pregnancy 
Outcomes through Maternity Care Coordination: A Systematic Review. Womens Health Issues. 
2016;26(1):87-99. 
15. 
Snaterse M, Dobber J, Jepma P, et al. Effective components of nurse-coordinated care to prevent 
recurrent coronary events: a systematic review and meta-analysis. Heart. 2016;102(1):50-56. 
16. 
Tricco AC, Antony J, Ivers NM, et al. Effectiveness of quality improvement strategies for 
coordination of care to reduce use of health care services: a systematic review and meta-analysis. 
CMAJ. 2014;186(15):E568-578. 
17. 
Weaver SJ, Che XX, Petersen LA, Hysong SJ. Unpacking Care Coordination Through a Multiteam 
System Lens: A Conceptual Framework and Systematic Review. Med Care. 2018;56(3):247-259. 
18. 
Craig C, Eby D, Whittington J. Care coordination model: better care at lower cost for people with 
multiple health and social needs. Cambridge MA: Institute for Healthcare improvement; 2011. 
19. 
Magnan S. Social determinants of health 101 for health care: five plus five. NAM Perspectives 
2017; October 9, 2017:9. Available at: https://nam.edu/social-determinants-of-health-101-for-
health-care-five-plus-five/. 
20. 
Daniel H, Bornstein SS, Kane GC, Health, Public Policy Committee of the American College of P. 
Addressing Social Determinants to Improve Patient Care and Promote Health Equity: An American 
College of Physicians Position Paper. Ann Intern Med. 2018;168(8):577-578. 
21. 
Katz A, Chateau D, Enns JE, et al. Association of the Social Determinants of Health With Quality of 
Primary Care. Ann Fam Med. 2018;16(3):217-224. 
22. 
Kaufman A. Theory vs Practice: Should Primary Care Practice Take on Social Determinants of 
Health Now? Yes. Ann Fam Med. 2016;14(2):100-101. 
23. 
Solberg LI. Theory vs Practice: Should Primary Care Practice Take on Social Determinants of 
Health Now? No. Ann Fam Med. 2016;14(2):102-103. 
24. 
Garg A, Boynton-Jarrett R, Dworkin PH. Avoiding the Unintended Consequences of Screening for 
Social Determinants of Health. JAMA. 2016;316(8):813-814. 
25. 
Williams DR, Costa MV, Odunlami AO, Mohammed SA. Moving upstream: how interventions that 
address the social determinants of health can improve health and reduce disparities. J Public 
Health Manag Pract. 2008;14 Suppl:S8-17. 
26. 
Soderberg K, Rajamani S, Wholey D, LaVenture M. Health Reform in Minnesota: An Analysis of 
Complementary Initiatives Implementing Electronic Health Record Technology and Care 
Coordination. Online J Public Health Inform. 2016;8(3):e204. 
27. 
Shippee ND, Finch M, Wholey DR. Assessing medical home mechanisms: certification, asthma 
education, and outcomes. Am J Manag Care. 2018;24(3):e79-e85. 
28. 
McDonald KM, Schultz E, Albin L, et al. Care Coordination Measures Atlas Version 4. Rockville, 
MD: AHRQ; 2014. 
29. 
McDonald KM, Sundaram V, Bravata DM, et al. Closing the Quality Gap: A Critical Analysis of 
Quality Improvement Strategies (Vol. 7: Care Coordination). Rockville (MD)2007. 
30. 
National Quality Forum. Priority Setting for Healthcare Performance Measurement: Addressing 
Performance Measure Gaps for the Health Workforce. Washington, DC: National Quality Forum; 
2014. 
31. 
National Quality Forum. NQF-Endorsed Measures for Care Coordination: Phase 3, 2014 (Technical 
Report). Washington, DC: National Quality Forum; 2014. 
32. 
National Quality Forum. Preferred Practices and Performance Measures for Measuring and 
Reporting Care Coordination. Washington, DC: National Quality Forum; 2010. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
63  
 
33. 
Schultz EM, Pineda N, Lonhart J, Davies SM, McDonald KM. A systematic review of the care 
coordination measurement landscape. BMC Health Serv Res. 2013;13:119. 
34. 
AHRQ. Care Coordination Measure for Primary Care Survey. Rockville MD, 2016. 
35. 
Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743-1748. 
36. 
McDonald KM, Schultz E, Chapman T, et al. Prospects for care coordination measurement using 
electronic data sources. Rockville, MD: Agency for Healthcare Research and Quality; 2012. AHRQ 
Publication No. 12-0014-EF. 
37. 
Solberg LI, Carlin CS, Peterson KA, Eder M. Certification as a medical home: does it make a 
difference in diabetes care. Ann Fam Med. 2019 (in review). 
38. 
HealthPartners White Paper: Total Cost of Care (TCOC) andTotal Resource Use: A Measurement 
Approach to Achieve the Triple Aim. 2017; 
https://www.healthpartners.com/ucm/groups/public/@hp/@public/documents/documents/dev
_057649.pdf. Accessed Dec 19, 2023. 
39. 
National Quality Forum. NQF Endorses Resource Use Measures. 2012; 
https://www.qualityforum.org/News_And_Resources/Press_Releases/2012/NQF_Endorses_Reso
urce_Use_Measures.aspx. Accessed Dec 19, 2023. 
40. 
National Quality Forum. Cost and Resource Use 2016-2017 Final Technical Report 2017; 
https://www.qualityforum.org/Publications/2017/08/Cost_and_Resource_Use_2016-
2017_Final_Technical_Report.aspx. Accessed Dec 19, 2023. 
41. 
Fowler FJ, Jr. Improving Survey Questions, Design, and Evaluation. Thousand Oaks CA: Sage 
Publications; 1995. 
42. 
Scholle SH, Vuong O, Ding L, et al. Development of and field test results for the CAHPS PCMH 
Survey. Med Care. 2012;50 Suppl:S2-10. 
43. 
Stucky BD, Hays RD, Edelen MO, Gurvey J, Brown JA. Possibilities for Shortening the CAHPS 
Clinician and Group Survey. Med Care. 2016;54(1):32-37. 
44. 
Elwyn G, Barr PJ, Grande SW, Thompson R, Walsh T, Ozanne EM. Developing CollaboRATE: a fast 
and frugal patient-reported measure of shared decision making in clinical encounters. Patient 
Educ Couns. 2013;93(1):102-107. 
45. 
Elwyn G, Thompson R, John R, Grande SW. Developing IntegRATE: a fast and frugal patient-
reported measure of integration in health care delivery. Int J Integr Care. 2015;15:e008. 
46. 
Fryer AK, Friedberg MW, Thompson RW, Singer SJ. patient perceptions of integrated care and 
their relationship to utilization of emergency, inpatient and outpatient services. Healthc (Amst). 
2017;5(4):183-193. 
47. 
Wisdom J, Creswell JW. Mixed Methods: Integrating Quantitative and Qualitative Data Collection 
and Analysis While Studying Patient-Centered Medical Home Models. AHRQ Publication No. 13-
0028-EF. Rockville, MD: Agency for Healthcare Research and Quality. , February 2013. 
48. 
Butler M, Epstein RA, Totten A, et al. AHRQ series on complex intervention systematic reviews-
paper 3: adapting frameworks to develop protocols. J Clin Epidemiol. 2017;90:19-27. 
49. 
von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening 
the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for 
reporting observational studies. J Clin Epidemiol. 2008;61(4):344-349. 
50. 
Tran VT, Montori VM, Eton DT, Baruch D, Falissard B, Ravaud P. Development and description of 
measurement properties of an instrument to assess treatment burden among patients with 
multiple chronic conditions. BMC Med. 2012;10:68. 
51. 
Billioux A, Verlander K, Anthony S, Alley D. Standardized screening for health-related social needs 
in clinical settings: the accountable health communities screening tool. Washington, DC: National 
Academy of Medicine; 2017. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
64  
 
52. 
Heinze G, Wallisch C, Dunkler D. Variable selection - A review and recommendations for the 
practicing statistician. Biom J. 2018;60(3):431-449. 
53. 
Patient-Centered Outcomes Research Institute. Principles for the Consideration of the Full Range 
of Outcomes Data in PCORI-Funded Research. 2021; https://www.pcori.org/research/about-our-
research/patient-centered-economic-outcomes/principles-consideration-full-range-outcomes-
data-pcori-funded-research. Accessed Dec 19, 2023. 
54. 
Pearl J. Causal diagrams for empirical research. Biometrika. 1995(82):669-710. 
55. 
Pearl J. Causality: Models, Reasoning, and Inference, 2nd ed. New York: Cambridge University 
Press; 2009. 
56. 
Hernán M, Robins J. Causal Inference. Boca Raton: Chapman & Hall/CRC, forthcoming; 2019. 
57. 
VanderWeele TJ, Ding P. Sensitivity Analysis in Observational Research: Introducing the E-Value. 
Ann Intern Med. 2017;167(4):268-274. 
58. 
Rowe JM, Rizzo VM, Shier Kricke G, et al. The Ambulatory Integration of the Medical and Social 
(AIMS) model: A retrospective evaluation. Soc Work Health Care. 2016;55(5):347-361. 
59. 
Steketee G, Ross AM, Wachman MK. Health Outcomes and Costs of Social Work Services: A 
Systematic Review. Am J Public Health. 2017;107(S3):S256-S266. 
60. 
Duan N, Wang Y. Heterogeneity of treatment effects. Shanghai Arch Psychiatry. 2012;24(1):50-
51. 
61. 
Hernan MA, Hernandez-Diaz S, Robins JM. A structural approach to selection bias. Epidemiology. 
2004;15(5):615-625. 
62. 
Cole SR, Hernan MA. Constructing inverse probability weights for marginal structural models. Am 
J Epidemiol. 2008;168(6):656-664. 
63. 
Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 
2006;3(2):77-101. 
64. 
Marshall C, Rossman GB. Designing Qualitative Research, 5th edition. 5th ed. Newbury Park: Sage 
Publications; 2011. 
65. 
Saldana J. The coding manual for qualitative researchers, 2nd edition. 2 ed. Thousand Oaks, CA: 
Sage Publications; 2013. 
66. 
Padgett DK. Qualitative methods in social work research, 3rd edition. 3 ed. Thousand Oaks, CA: 
Sage Publications Ltd.; 2017. 
67. 
Lincoln YS, Guba EG. Naturalistic Inquiry. Newbury Park, CA: Sage; 1985. 
68. 
Morse J, Barrett M, Mayan M, Olson K, Spiers J. Verification strategies for establishing reliability 
and validity in qualitative research. International Journal of Qualitative methods. 2002;1(2):13-22. 
69. 
Rudolph KE, Stuart EA. Using Sensitivity Analyses for Unobserved Confounding to Address 
Covariate Measurement Error in Propensity Score Methods. Am J Epidemiol. 2018;187(3):604-
613. 
70. 
Vanderweele TJ, Arah OA. Bias formulas for sensitivity analysis of unmeasured confounding for 
general outcomes, treatments, and confounders. Epidemiology. 2011;22(1):42-52. 
71. 
Elwyn G, Cochran N, Pignone M. Shared Decision Making-The Importance of Diagnosing 
Preferences. JAMA Intern Med. 2017. 
72. 
MN Community Measurement. Direct Data Submission (DDS) Resources.  
https://helpdesk.mncm.org/helpdesk/KB/Category/394778-direct-data-submission-dds-
resources. Accessed May 2, 2023. 
73. 
MN Community Measurement. Data Collection Technical Guide. Direct Data Submission (DDS) 
Data Portal: Optimal Asthma Control Asthma Education and Self-Management. Minneapolis, 
2022. 
74. 
MN Community Measurement. Data Collection Technical Guide. Direct Data Submission (DDS) 
Data Portal: Colorectal Cancer Screening. Minneapolis, 2022. 

PCORI Protocol IHS-2019C1-15625 
Version 1.3, January 29, 2024 
 
65  
 
75. 
MN Community Measurement. Data Collection Technical Guide. Direct Data Submission (DDS) 
Data Portal: Depression Care Measures. Minneapolis, 2022. 
76. 
MN Community Measurement. Data Collection Technical Guide. Direct Data Submission (DDS) 
Data Portal: Optimal Diabetes Care. Minneapolis, 2022. 
77. 
MN Community Measurement. Data Collection Technical Guide. Direct Data Submission (DDS) 
Data Portal: Optimal Vascular Care. Minneapolis, 2022. 
 
 

PCORI Care Coordination  
Version 0.1 
Protocol IHS-2019C1-15625 
April 23, 2021 
Version 1.3, January 29, 2024 
 
66  
 
17 Appendices  
The below appendices represent currently developed study materials. Materials for future data 
collection events or any future changes to these instruments will be updated as amendments to the 
protocol once IRB approved.  
 
 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
67  
 
17.1 Appendix A: Patient data specifications  
These data specifications are for identification of patient cohorts as described in Section 5.1.1. Cohort-specific requirements are identified 
throughout. Data were collected at two separate timepoints most proximal to each cohort’s care coordination index date range.  
 
Clinical Data Preparations  
Step 1:  Identify those patients 18 years of age and older who had a first-ever care coordination start/enrollment date in your care system as 
follows:  
• 
Historical Cohort:  patients with start/enrollment date between 01/01/2018 to 02/28/2019 
• 
Primary Cohort:  patients with start/enrollment date between 01/01/2021 to 12/31/2021 
Step 2:  Pull the clinical data elements from your electronic systems and prepare two data files as specified in the tables below. Submit the data 
files during the following data submission periods by cohort: 
• 
Historical Cohort data submission period:  06/14/2021 to 08/15/2021 
• 
Primary Cohort data submission period:  3/15/2022 to 5/15/2022 
Care Coordination Definitions   
Please use the guidance below for identifying patients for the study.  
 Historical Cohort Enrollment Start Date: Please include patients whose first-ever care coordination enrollment started in your care 
system between 1/1/2018 – 2/28/2019. Please exclude patients whose care coordination began within your care system before 
1/1/2018. 
 Primary Cohort Enrollment Start Date: Please include patients whose first-ever care coordination enrollment started in your care 
system between 01/01/2021 – 12/31/2021. Please exclude patients whose care coordination began within your care system before 
01/01/2021. Please include patients whose status is deceased but who were enrolled in care coordination during the specified time.  
 Time in Care Coordination: The study is intended to include patients who were enrolled in care coordination and engaged for a 
meaningful period of time, using whatever criteria make the most sense in your care system. Please do not include patients who were 
temporarily or “partially” enrolled. For example, some medical groups may apply an inclusion criterion to their query such as: include 
only patients who were enrolled for at least 2 months with at least 3 care coordination encounters within the care system. You may also 
use other ways to narrow the list to those enrolled patients intended for the study, depending on what parameters are used to define a 
“fully enrolled” care coordination patient in your system. 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
68  
 
 Exclusions: Please apply any appropriate research exclusion lists to exclude patients who have opted out of research in your 
organization. 
Summary of Changes for Primary Cohort 
We had hoped to have few changes for this data submission but concluded that expanding the number of data points for three quality measures 
to include the following data would greatly improve the ability to measure changes in care quality from care coordination. If you have any 
questions or concerns about adding these data, please contact support@mncm.org. 
• 
File 3:  All HbA1c tests obtained during the specified time period (see below). 
• 
File 4:  All blood pressures obtained during the specified time period (see below). 
• 
File 5:  All height and weight values obtained during the specified time period (see below). Used for calculating BMI. 
Questions?  Please visit the MNCARES Information Page for answers to common questions or contact us at support@mncm.org.  
 
Data Elements & Field Specifications – Files 1-5 
File 1:  Patient Demographic, Care Coordination Enrollment, Encounter Data 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
A 
Patient ID 
Enter the patient ID that is submitted for clinical quality data submission 
to MNCM. If your clinic has not submitted MRNs or a unique patient ID in 
past submissions, please contact support@mncm.org  
• 
Unique patient identifier for clinic. 
• 
DO NOT enter an SSN. 
R 
String; up to 
50 characters 
Blank fields 
B 
Patient First 
Name 
Enter the first name of the patient.  
R 
Text 
Blank fields 
C 
Patient Middle 
Name 
Please enter the patient’s middle name or initial if it is recorded in the 
medical record.  
S 
Text 
 
D 
Patient Last 
Name 
Enter the last name of the patient.  
R 
Text 
Blank fields 
E 
Patient Master 
Index (PMI) 
This is a unique patient identification number that the Department of 
Human Services uses for previous and current Minnesota Health Care 
Program participants.  
S 
String; up to 
50 characters 
 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
69  
 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
F 
Current 
Primary 
Insurance 
Member ID 
Enter the most recent Member ID on file as of the date of the data pull.  
• 
Unique patient identifier for health plan 
• 
Do NOT enter an SSN; instead, enter “999”  
R 
String; up to 
50 characters 
Blank fields 
G 
Current 
Primary 
Insurance 
Enter the most recent primary insurance on file as of the date of the data 
pull.  
Please refer to a separate document entitled Insurance Coverage Data 
Elements, Field Specifications &Codes for field specifications. 
R 
Number; up 
to 2 digits 
Blank fields 
Values 
outside 
allowable 
range 
H 
Current 
Subscriber 
name 
Enter the full name for the person that subscribes to the health plan.  
S 
Text 
 
I 
Prior Primary 
Insurance 
Member ID 
Enter the previous Member ID on file if obtainable.  
• 
Unique patient identifier for health plan 
• 
Do NOT enter an SSN; instead, enter “999”.  
S 
String; up to 
50 characters 
 
J 
Prior Primary 
Insurance 
Please refer to a separate document entitled Insurance Coverage Data 
Elements, Field Specifications &Codes for field specifications. 
 
S 
Number; up 
to 2 digits 
Values 
outside 
allowable 
range 
K 
Prior 
Subscriber 
name 
Enter the full name for the person that subscribed to the prior health plan.  
S 
Text 
 
L 
Patient Date of 
Birth (DOB) 
Must be age 18 years or older as of the start/enrollment date of care 
coordination (per AG below). 
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
M 
Patient Sex 
F = Female 
M = Male 
U = Unknown/Undefined 
R 
Text; 1 
character 
Blank fields 
Values 
outside 
allowable 
range 
N 
Patient Date of 
Death (DOD) 
 
S 
mm/dd/yyyy 
or 
m/d/yyyy 
 
O 
Patient Status 
Enter the most recent patient status at the time of the data pull.  
0 = Deceased 
1 = Alive 
S 
Number; 1 
digit 
 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
70  
 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
P 
Race1 
Enter the code that corresponds to the patient reported race. For patients 
who report more than one race, enter one code per field for each reported 
race, up to five. Do not submit the same code in multiple fields. 
1 = American Indian or Alaska Native 
2 = Asian 
3 = Black or African American 
5 = Native Hawaiian/Other Pacific Islander 
6 = White 
7 = Some other race/Patient does not identify with any of the race 
categories provided. 
97 = Patient actively chose not to disclose/declined 
98 = Patient reports that race is unknown. 
If patient was not asked for their race or if race was left blank by patient, 
leave the fields blank. 
S 
Number; up 
to 2 digits 
Values 
outside 
allowable 
range 
Q 
Race2 
S 
Number; up 
to 2 digits 
R 
Race3 
S 
Number; up 
to 2 digits 
S 
Race4 
S 
Number; up 
to 2 digits 
T 
Race5 
S 
Number; up 
to 2 digits 
U 
Ethnicity 
Enter the code that corresponds to the patient-reported ethnicity 
4 = Hispanic or Latino 
8 = Not Hispanic or Latino 
97 = Patient actively chose not to disclose/declined 
98 = Patient reports that ethnicity is unknown 
If patient was not asked for their ethnicity or if ethnicity was left blank by 
patient, leave the field blank. 
S 
Number; up 
to 2 digits 
Values 
outside 
allowable 
range 
V 
Preferred 
Language 
Enter the code that corresponds to the patient-reported preferred 
language. Please refer to a separate document entitled RELC Data 
Elements, Field Specifications & Codes for coding table. Additional options 
include: 
97 = Patient actively chose not to disclose/declined 
98 = Patient reports that preferred language is unknown. 
99 = Patient reported preferred language does not match one of the 
available codes. Enter name of preferred language in Preferred 
Language Other field. 
If patient was not asked for their preferred language or if preferred 
language was left blank by patient, leave the fields blank. 
S 
Number; up 
to 2 digits 
Values 
outside 
allowable 
range 
W 
Preferred 
Language Other 
If Element Position V = 99, submit preferred language. 
S 
String; up to 
50 characters 
 
X 
Country of 
Origin 
Enter the code that corresponds to the patient-reported country of origin. 
Please refer to a separate document entitled RELC Data Elements, Field 
Specifications & Codes for coding table. Additional options include: 
S 
Number; up 
to 3 digits 
Values 
outside 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
71  
 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
997 = Patient actively chose not to disclose/declined 
998 = Patient reports that country of origin is unknown. 
999 = Patient reported country of origin does not match one of the 
available codes. Enter name of country of origin in Country of Origin 
Other field. 
If patient was not asked for their country of origin or if country of origin 
was left blank by patient, leave the fields blank. 
allowable 
range 
Y 
Country of 
Origin Other 
If Element Position X = 999, submit country of origin. 
S 
String; up to 
50 characters 
 
Z 
Street Address 
Patient’s primary residence 
R 
String; up to 
50 characters 
Blank fields 
AA 
City 
Patient’s primary residence 
R 
String; up to 
50 characters 
Blank fields 
 
AB 
State 
Standard two-character state abbreviation 
Patient’s primary residence 
 
R 
Text; 2 
characters 
Blank fields 
 
AC 
ZIP Code 
Minimum of five digits 
Patient’s primary residence 
R 
Number 
Blank fields 
Values with 
less than five 
digits 
AD 
Primary Phone 
Number 
Minimum of 10 digits 
R 
Number 
 
AE 
Secondary 
Phone Number 
Minimum of 10 digits 
S 
Number 
 
AF 
Interpreter 
needed? 
0 = No 
1 = Yes 
S 
Number 
 
AG 
Start/enrollmen
t date of care 
coordination  
Enter the date that corresponds to the start of the patient’s care 
coordination enrollment between 1/1/2018-2/28/2019 (Historical 
Cohort) and 1/1/2021 – 12/31/2021 (Primary Cohort). 
• 
The start date should represent the patient’s first-ever 
enrollment in care coordination in your care system 
• 
Exclude patients whose care coordination began within your 
care system before 1/1/2018 (Historical Cohort) or 1/1/2021 
(Primary Cohort). 
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
72  
 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
• 
Only include care coordination patients that were engaged for a 
meaningful period of time as defined as Time in Care 
Coordination above.  
AH 
Clinic ID from 
the start of care 
coordination 
enrollment 
Enter the MNCM-assigned clinic ID associated with the start of the 
patient’s care coordination enrollment.  
R 
Number; up 
to 4 digits 
Blank fields 
 
AI 
Date of most 
recent care 
coordination 
encounter 
• 
Enter the date that corresponds with the patient’s most recent 
care coordination encounter. This date can occur any time up 
until the data is pulled.  
• 
If patient did not have any subsequent care coordination 
encounters after the initial start/enrollment encounter, enter 
the start/enrollment care coordination date from AG. 
Type of encounter may be any of the following: office, phone, video, or 
home visit. 
S 
mm/dd/yyyy 
or 
m/d/yyyy 
 
AJ 
Clinic ID from 
the most recent 
care 
coordination 
encounter 
Enter the MNCM-assigned clinic ID associated with the patient’s most 
recent care coordination encounter. 
S 
Number; up 
to 4 digits 
 
AK 
Date of most 
recent 
encounter  
• 
Enter the most recent ambulatory encounter date, regardless of 
whether the visit was a care coordination visit. This date can 
occur any time up until the data is pulled.  
• 
If patient did not have any subsequent encounters after the 
initial 
start/enrollment 
encounter, 
enter 
the 
date 
of 
start/enrollment care coordination date from AG. 
• 
Type of encounter may be any of the following: office, phone, 
video, or home visit. 
S 
mm/dd/yyyy 
or 
m/d/yyyy 
 
AL 
Clinic ID from 
most recent 
encounter 
Enter the MNCM-assigned clinic ID associated with the most recent 
ambulatory encounter.  
S 
Number; up 
to 4 digits 
 
AM 
Count of care 
coordination 
encounters 
• 
Count of all care coordination encounters between the patient’s 
start date (AG) and most recent care coordination encounter 
(AI).  
• 
This count should include the initial start/enrollment date and 
the most recent care coordination encounter. If patient did not 
S 
Number; up 
to 3 digits 
 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
73  
 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
have 
any 
subsequent 
encounters 
after 
the 
initial 
start/enrollment encounter, enter “1”. 
• 
Type of encounter may be any of the following: office, phone, 
video, or home visit. 
* Both Required (R) and Situational (S) data are relevant to and important for the study. Required data must be submitted and cannot be blank. 
Situational data is submitted if the clinic collects and can extract or obtain the information from their record system. Submit data for those 
patients when the information is available (e.g., secondary phone number). If the data was not collected or is not obtainable, the field can be left 
blank.  
 
File 2:  Diagnosis Codes from Patient’s Active Problem List 
* Both Required (R) and Situational (S) data are relevant to and important for the study. Required data must be submitted and cannot be blank. 
Situational data is submitted if the clinic collects and can extract or obtain the information from their record system. Submit data for those 
patients when the information is available (e.g., secondary phone number). If the data was not collected or is not obtainable, the field can be left 
blank.  
 
 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
A 
Patient ID 
Enter the Patient ID that was submitted in File 1. 
• 
Unique patient identifier for clinic. 
• 
DO NOT enter an SSN. 
R 
String; up to 
50 characters 
Blank fields 
B 
Clinic ID from 
the start of care 
coordination 
enrollment 
Enter the MNCM-assigned clinic ID associated with the start of the 
patient’s care coordination enrollment. This is the same ID entered in 
the Demographic file, element order AH. 
R 
Number; up 
to 4 digits 
Blank fields 
 
C-AZ 
Diagnosis Codes 
from Patient’s 
Active Problem 
List 
Enter all diagnosis codes (e.g., ICD-10) associated with the patient’s 
active problem list. This includes diagnoses unrelated to the care 
coordination.  
• 
All applicable characters, including decimals (e.g., E11.9) 
• 
Up to 50 diagnoses may be submitted 
• 
One code per field 
R 
String; up to 
50 characters 
Blank fields 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
74  
 
File 3 (Primary Cohort only):  HbA1c Tests 
Contains all tests obtained or documented in the medical record from 01/01/2020 through the date of the data pull. One test per row. Tests 
from outside providers that are documented in the patient record may also be included.   
* Both Required (R) and Situational (S) data are relevant to and important for the study. Required data must be submitted and cannot be blank. 
Situational data is submitted if the clinic collects and can extract or obtain the information from their record system. Submit data for those 
patients when the information is available (e.g., secondary phone number). If the data was not collected or is not obtainable, the field can be left 
blank.  
 
 
 
 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
A 
Patient ID 
Enter the Patient ID that was submitted in File 1. 
• 
Unique patient identifier for clinic. 
• 
DO NOT enter an SSN. 
R 
String; up to 
50 characters 
Blank fields 
B 
HbA1c Date 
Enter the date of the HbA1c test.  
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
C 
HbA1c Value 
Enter the value of the HbA1c test result. Include decimals. If the result 
was too high to calculate, leave blank.  
S 
Number 
 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
75  
 
File 4 (Primary Cohort only):  Blood Pressures 
Contains all blood pressures obtained or documented in the medical record from 01/01/2020 through the date of the data pull. Include readings 
from eligible specialties below. For multiple readings at one encounter, submit all recorded readings, one per row.  Results from outside 
providers that are documented in the patient record may also be included. Blood pressures that are taken by the patient on a digital device in 
the context of a virtual (online or telephone) visit are acceptable.  
Eligible Specialties: 
• 
Family Medicine 
• 
Internal Medicine 
• 
Geriatric Medicine 
• 
Cardiology 
• 
Endocrinology 
Do not include results: 
• 
Taken during an acute inpatient stay or an ED visit. 
• 
Taken during an outpatient visit which was for the sole purpose of having a diagnostic test or surgical procedure performed. 
• 
Taken the same day a major diagnostic or surgical procedure. 
* Both Required (R) and Situational (S) data are relevant to and important for the study. Required data must be submitted and cannot be blank. 
Situational data is submitted if the clinic collects and can extract or obtain the information from their record system. Submit data for those 
patients when the information is available (e.g., secondary phone number). If the data was not collected or is not obtainable, the field can be left 
blank.  
 
 
 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
A 
Patient ID 
Enter the Patient ID that was submitted in File 1. 
• 
Unique patient identifier for clinic. 
• 
DO NOT enter an SSN. 
R 
String; up to 
50 characters 
Blank fields 
B 
Blood Pressure 
Date 
Enter the date of the blood pressure result.  
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
C 
Systolic Value 
Enter the value of the systolic reading.  
R 
Number 
Blank fields 
D 
Diastolic Value 
Enter the value of the diastolic reading 
R 
Number 
Blank fields 

Appendix A 
 
MNCARES Patient data specifications 
 
Version 1.3, January 29, 2024 
 
76  
 
File 5 (Primary Cohort only):  Height and Weight 
Contains all heights and weights obtained or documented in the medical record from 01/01/2020 through the date of the data pull. One height 
and weight per row.   
* Both Required (R) and Situational (S) data are relevant to and important for the study. Required data must be submitted and cannot be blank. 
Situational data is submitted if the clinic collects and can extract or obtain the information from their record system. Submit data for those 
patients when the information is available (e.g., secondary phone number). If the data was not collected or is not obtainable, the field can be left 
blank.  
 
 
 
 
 
Element 
Order 
Field Name 
Details 
Required or 
Situational* 
Format/Field 
Length 
Error Causes 
A 
Patient ID 
Enter the Patient ID that was submitted in File 1. 
• 
Unique patient identifier for clinic. 
• 
DO NOT enter an SSN. 
R 
String; up to 
50 characters 
Blank fields 
B 
Encounter Date 
Enter the date of the encounter where height/weight was obtained by 
clinical staff.   
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
C 
Height 
Enter the value of the patient’s height (inches, including decimals if 
applicable).  
S 
Number 
 
D  
Weight 
Enter the value of the patient’s weight (pounds, including decimals if 
applicable).  
S 
Number 
 

Appendix B 
 
MNCARES Care quality specifications 
Version 1.2, May 1, 2023 
 
77 
17.2 Appendix B: Care quality data specifications  
Care quality measure specifications are for use in measuring care quality outcomes as described in Section 5.1.2. Data were collected for 
measurement years 2017-2022 in order to cover the pre-post measurement for both cohorts. Up to date measurement specifications can be found 
online in MN Community Measurement’s Knowledge Base. 72  The below table describes the basic description, eligible population, and exclusions 
that guide clinical data submission to MNCM.  
Optimal Asthma Control adapted from Optimal Asthma Control Measure Specifications, Measurement Year 2022 73 
Description 
Eligible Population 
Required exclusions 
The percentage of pediatric (5-17 years of 
age) and adult (18-50 years of age) patients 
who had a diagnosis of asthma and whose 
asthma was optimally controlled during the 
measurement period as defined by 
achieving BOTH of the following: 
• 
Asthma well-controlled as defined by 
the most recent asthma control tool 
result available during the 
measurement period 
• 
Patient not at elevated risk of 
exacerbation as defined by less than 
two emergency department visits 
and/or hospitalizations due to asthma 
in the last 12 months 
Separate rates are reported for each age 
group. 
Eligible Specialties: Family Medicine, Internal Medicine, Pediatrics, 
Allergy/Immunology, Pulmonology 
Eligible providers:  Medical Doctor (MD), Doctor of Osteopathy (DO), 
Physician Assistant (PA), Advanced Practice Registered Nurses (APRN) 
Ages: 5 years or older at the start of the measurement period AND less 
than 51 years at the end of the measurement period 
Diagnosis: Patient had a diagnosis of asthma (Asthma Value Set) with any 
contact during the current or prior measurement period OR had asthma 
(Asthma Value Set) present on an active problem list at any time during 
the measurement period.  
Both contacts AND the active problem list must be queried for diagnosis 
(Asthma Value Set). 
Event: At least one established patient office or telehealth visit 
(Established Pt Asthma Value Set) performed or supervised by an eligible 
provider in an eligible specialty for any reason during the measurement 
period 
• Patient had a diagnosis of cystic 
fibrosis, COPD, emphysema, or 
acute respiratory failure 
(Obstructive Lung and Respiratory 
Failure Value Set) 
Colorectal Cancer Screening adapted from Colorectal Cancer Screening Measure Specifications, Measurement Year 2022 74 
Description 
Eligible Population 
 
Required exclusions 
The percentage of patients 45-75 years of 
age who had appropriate screening for 
colorectal cancer 
Eligible Specialties: Family Medicine, Internal Medicine, Geriatric 
Medicine, Ostetrics/Gynecology  
Eligible providers:  Medical Doctor (MD), Doctor of Osteopathy (DO), 
Physician Assistant (PA), Advanced Practice Registered Nurses (APRN) 
• Patient was in hospice or 
receiving palliative care at any 
time during the measurement 
period (Palliative Care Value Set) 

Appendix B 
 
MNCARES Care quality specifications 
Version 1.2, May 1, 2023 
 
78 
Ages: 45 years or older at the start of the measurement period AND less 
than 76 years at the end of the measurement period 
Event: At least one established patient visit (CRC Screening Visit Value 
Set) performed or supervised by an eligible provider in an eligible 
specialty for any reason during the measurement period.  
Depression Care adapted from Depression Care Measures, Measurement Year 2022 75 
Description 
Eligible Population 
Required exclusions 
The percentage of adolescent patients (12 
to 17 years of age) and adult patients (18 
years of age and older) with Major 
Depression or Dysthymia who have 
completed a PHQ-9 or PHQ-9M tool during 
the applicable four-month measurement 
period in which there was a qualifying 
encounter. 
Eligible Specialties for diagnosing Depression/Dysthymia: Family 
Medicine, Internal Medicine, Geriatric Medicine, Psychiatry, Behavioral 
Health, Pediatric/Adolescent Medicine 
Eligible providers for diagnosing Depression/ Dysthymia: Medical 
Doctor (MD), Doctor of Osteopathy (DO), Physician Assistant (PA), 
Advanced Practice Registered Nurses (APRN) These providers are also 
eligible, if supervised by a physician: Licensed Psychologist (LP), Licensed 
Independent Clinical Social Worker (LICSW), Licensed Professional Clinical 
Counselor (LPCC), Licensed Marriage & Family Therapist (LMFT) 
Ages: 12 years or older at the time of the qualifying encounter 
Event: Patients with an encounter* coded with Major Depression or 
Dysthymia (Major Depression or Dysthymia Value Set) during the specific 
measurement period *For this measure, an encounter includes but is not 
limited to any of the following: office visit, psychiatry, or psychotherapy 
visit, telephone, or online encounter. 
• Patient had a diagnosis of 
Bipolar Disorder (Bipolar Disorder 
Value Set) any time prior to the 
end of the measurement period • 
Patient had an active diagnosis of 
Schizophrenia or Psychotic 
Disorder (Schizophrenia Psychotic 
Disorder Value Set) any time prior 
to the end of the measurement 
period 
  
Optimal Diabetes Care adapted from Optimal Diabetes Care Measure, Measurement Year 2022 76 
Description 
Eligible Population 
Required exclusions 
The percentage of patients 18-75 years of 
age who had a diagnosis of type 1 or type 2 
diabetes and whose diabetes was optimally 
managed during the measurement period 
as defined by achieving ALL of the 
following:  
• HbA1c less than 8.0 mg/dL  
• Blood pressure less than 140/90 mmHg  
Eligible Specialties: Family Medicine, Internal Medicine, Geriatric 
Medicine, Endocrinology 
Eligible providers:  Medical Doctor (MD), Doctor of Osteopathy (DO), 
Physician Assistant (PA), Advanced Practice Registered Nurses (APRN) 
Ages: 18 years or older at the start of the measurement period AND less 
than 76 years at the end of the measurement period 
none 

Appendix B 
 
MNCARES Care quality specifications 
Version 1.2, May 1, 2023 
 
79 
• On a statin medication, unless allowed 
contraindications or exceptions are present  
• Non-tobacco user  
• Patient with ischemic vascular disease is 
on daily aspirin or antiplatelets, unless 
allowed contraindications or exceptions are 
present 
Diagnosis: Patient had a diagnosis of diabetes (Diabetes Value Set) with 
any contact during the current or prior measurement period OR had 
diabetes (Diabetes Value Set) present on an active problem list at any 
time during the measurement period. Both contacts AND the active 
problem list must be queried for diagnosis (Diabetes Value Set). 
Event: At least one established patient office or telehealth visit 
(Established Pt Diabetes & Vasc Value Set) performed or supervised by an 
eligible provider in an eligible specialty for any reason during the 
measurement period 
Optimal Vascular Care adapted from Optimal Vascular Care Measure, Measurement Year 2022 77 
Description 
Eligible Population 
Required exclusions 
The percentage of patients 18-75 years of 
age who had a diagnosis of ischemic 
vascular disease (IVD) and whose IVD was 
optimally managed during the 
measurement period as defined by 
achieving ALL of the following:  
• Blood pressure less than 140/90 mmHg  
• On a statin medication, unless allowed 
contraindications or exceptions are present  
• Non-tobacco user  
• On daily aspirin or anti-platelets, unless 
allowed contraindications or exceptions are 
present 
Eligible Specialties: Family Medicine, Internal Medicine, Geriatric 
Medicine, Cardiology 
Eligible providers:  Medical Doctor (MD), Doctor of Osteopathy (DO), 
Physician Assistant (PA), Advanced Practice Registered Nurses (APRN) 
Ages: 18 years or older at the start of the measurement period AND less 
than 76 years at the end of the measurement period 
Diagnosis: Patient had a diagnosis of ischemic vascular disease (Ischemic 
Vascular Disease Value Set) with any contact during the current or prior 
measurement period OR had ischemic vascular disease (Ischemic 
Vascular Disease Value Set) present on an active problem list at any time 
during the measurement period. Both contacts AND the active problem 
list must be queried for diagnosis (Ischemic Vascular Disease Value Set). 
Event: At least one established patient office or telehealth visit 
(Established Pt Diabetes & Vasc Value Set) performed or supervised by an 
eligible provider in an eligible specialty for any reason during the 
measurement period. 
none 

Appendix C 
MNCARES Claims data specifications  
 
Version 1.2, May 1, 2023 
80 
17.3 Appendix C: Claims data specifications  
These data specifications are for collection of Utilization Outcomes data as described in Section 5.1.3 
 
General Rules for Administrative Claim files: 
• 
Date Range: One-year pre and post enrollment in Health Care Homes   
• 
Time period is based on discharge date when applicable 
• 
Payers should submit paid claims and final claims only (exclude denied claims) 
• 
Partial facility claims are acceptable if the full claim is not available 
• 
Notes: Medicare Cost Plan patients will not have hospital claims available 
• 
Note: Pharmacy claims are not always available for commercial patients 
• 
TBD: multiple payers for same patient 
• 
Note: dollar amounts are for RVU study, not for a cost comparison  
Summary of changes (March 2023): 
• Added product type code 77 for Minnesota Senior Care Plus (MSC+); see page 2, Enrollment Data File, Element Order I  
Enrollment Data File 
Submit one record per patient per enrollment period. Patients may be listed in multiple rows if the enrollment was not continuous.  
Element 
Order 
Who is 
responsible for 
supplying data? 
Field Name 
Details 
Required or 
Situational 
Format/Field 
Length 
Error Causes 
A 
MNCARES 
Current 
Primary 
Insurance 
Member ID 
• 
Unique patient identifier for health plan 
• 
Do NOT use SSN.  
R 
String; up to 50 
characters 
Blank fields 
B 
MNCARES 
Study ID 
• 
Unique MNCARES assigned non-PHI 
study ID for patient. 
R 
String; up to 10 
characters 
Blank fields 
C 
MNCARES 
Patient Master 
Index 
• 
From Clinic Data File 
S 
 
 

Appendix C 
MNCARES Claims data specifications  
 
Version 1.2, May 1, 2023 
81 
Element 
Order 
Who is 
responsible for 
supplying data? 
Field Name 
Details 
Required or 
Situational 
Format/Field 
Length 
Error Causes 
D 
MNCARES 
Medical Record 
Number 
• 
Unique patient identifier for clinic EMR 
• 
Do NOT use SSN. 
R 
String; up to 50 
characters 
Blank fields 
E 
MNCARES 
Patient First 
Name 
Enter the first name of the patient.  
R 
String 
Blank fields 
Values outside 
allowable range 
F 
MNCARES 
Patient Middle 
Name 
Enter the middle name of the patient if 
available.  
S 
String 
 
G 
MNCARES 
Patient Last 
Name 
Enter the last name of the patient.  
R 
String 
Blank fields 
Values outside 
allowable range 
H 
MNCARES 
Patient Date of 
Birth (DOB) 
 
R 
mm/dd/yyyy or 
m/d/yyyy 
Blank fields 
I 
Health Plan 
 
Product Type 
• 
1 = Commercial  
• 
5A = Medicare Advantage  
• 
5C = Medicare Cost 
• 
6 = MSHO 
• 
7A = Medicaid (MNCare) 
• 
7B = Medicaid (PMAP) 
• 
7C= Medicaid (Hennepin Health) 
• 
8 = SNBC (Special Needs Basic Care)  
• 
77 = Minnesota Senior Care Plus (MSC+) 
R 
String; 2 digits 
Blank fields 
J 
Health Plan 
Insurance Plan 
Enrollment 
Start Date 
 
R 
mm/dd/yyyy or 
m/d/yyyy 
Blank fields 
K 
Health Plan 
Insurance Plan 
Enrollment End 
Date 
 
R 
mm/dd/yyyy or 
m/d/yyyy 
Blank fields 
L 
Health Plan 
Pharmacy Data 
availability 
• 
0=No 
• 
1=Yes 
• 
Is pharmacy data available in the plan 
data warehouse? 
R 
String; 1 digit 
Blank fields 
 

Appendix C 
MNCARES Claims data specifications  
 
Version 1.2, May 1, 2023 
82 
Professional Data File (HCFA 1500) 
Standard claim form for physicians and other health care professionals. 
Submit one line per patient per procedure claim. Patients may be listed in multiple rows for multiple procedure claims.  
Element 
Order 
Field Name 
Details 
Required or 
Situational 
Format/Field 
Length 
Error Causes 
A 
Study ID 
Unique MNCARES assigned non-PHI study ID for 
patient 
R 
String; up to 10 
characters 
Blank fields 
B 
Claim ID 
 
 
R 
 
Blank fields 
C 
Provider First Name 
Enter the first name of the provider.  
S 
String 
 
D 
Provider Middle Name 
Enter the middle name of the provider if it is recorded 
in the medical record.  
S 
String 
 
E 
Provider Last Name or 
Facility Name 
Enter the last name of the provider.  
R 
String 
Blank fields 
Values outside 
allowable range 
F 
Provider Specialty Type 
Enter the code that corresponds to the Provider’s 
Specialty. Please refer to the  
Standard Medicare Taxonomy Codes. 
R 
String; up to 15 
characters 
Blank fields 
Values outside 
allowable range 
G 
Clinic Site of Service 
 
R 
Number; up to 
2 digits 
Blank fields 
Values outside 
allowable range 
H 
Start Date of service 
 
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
I 
End Date of service 
 
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
J 
Procedure Code 
Enter the CPT/HCPCS code that corresponds to the 
procedure claim. Do not include modifiers. 
R 
String; 
5 
characters 
Blank fields 
K 
Modifier(s) 
 
S 
String; 
8 
characters 
 
L 
Units 
 
R 
 
Blank fields 
M 
Billed Charges 
To be used only for calculation of Relative Value 
R 
Number, 
2 
digits 
 

Appendix C 
MNCARES Claims data specifications  
 
Version 1.2, May 1, 2023 
83 
Facility Data File (UB04) 
Hospital and Facility Header File, one line per admission or bundle of outpatient services. 
Element 
Order 
Field Name 
Details 
Required or 
Situational 
Format/Field 
Length 
Error Causes 
A 
Study ID 
Unique MNCARES assigned non-PHI study ID for 
patient 
R 
String; up to 10 
characters 
Blank fields 
B 
Claim ID 
 
R 
 
Blank fields 
C 
Facility Name 
Facility Name  
R 
String 
Blank fields 
Values outside 
allowable range 
D 
Provider Specialty Type 
Enter the code that corresponds to the Provider’s 
Specialty. Please refer to the  
Standard Medicare Taxonomy Codes. 
R 
 
 
E 
Bill Type 
 
R 
String, 3 digits 
 
F 
DRG 
Inpatient only 
S 
String, 
5 
characters 
 
G 
DRG Grouper 
1=MS DRG 
2=APR DRG 
3=All Other 
R 
String, 
1 
character 
 
H 
Admission Date 
 
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
I 
Discharge Date 
 
 
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
J 
Billed Charges 
To be used only for calculation of Relative Value 
R 
Number, 
2 
digits 
 
 
 
 

Appendix C 
MNCARES Claims data specifications  
 
Version 1.2, May 1, 2023 
84 
UB04 Detail File: Outpatient claims only 
Element 
Order 
Field Name 
Details 
Required or 
Situational 
Format/Field 
Length 
Error Causes 
A 
Study ID 
Unique MNCARES assigned non-PHI study ID for 
patient 
R 
String; up to 10 
characters 
Blank fields 
B 
Claim ID 
 
R 
 
Blank fields 
C 
Revenue Code 
 
R 
String, 3 digits 
Blank fields 
D 
Procedure Code 
Enter the CPT/HCPCS code that corresponds to the 
procedure claim.   
S 
String, 5 digits 
 
E 
Units 
 
R 
Number, 
no 
digits 
 
F 
Billed Charges 
To be used only for calculation of Relative Value 
R 
Number, 
2 
digits 
 
 
Diagnosis Data File (HCFA and UB) 
ICD10 diagnosis codes from both professional and facility claims. One line per diagnosis per claims.  No limit to the number of diagnosis codes 
per claim. 
Element 
Order 
Field Name 
Details 
Required or 
Situational 
Format/Field 
Length 
Error Causes 
A 
Study ID 
Unique MNCARES assigned non-PHI study ID for 
patient 
R 
String; up to 10 
characters 
Blank fields 
B 
Claim ID 
 
 
R 
 
Blank fields 
C 
ICD-10 Diagnosis 
All applicable characters, including decimals, MUST be 
included. One record per diagnosis code per claim. 
 
R 
String; up to 9 
characters 
Blank fields 
 
 
 

Appendix C 
MNCARES Claims data specifications  
 
Version 1.2, May 1, 2023 
85 
ICD10 Procedure Data File (for UB04) 
ICD10 procedure codes from inpatient UB04 claims. No limit to the number of procedures per claim. 
Element 
Order 
Field Name 
Details 
Required or 
Situational 
Format/Field 
Length 
Error Causes 
A 
Study ID 
Unique MNCARES assigned non-PHI study ID for 
patient 
R 
String; up to 10 
characters 
Blank fields 
B 
Claim ID 
 
R 
 
Blank fields 
C 
ICD-10 Procedure Code 
All applicable characters, including decimals, MUST be 
included. One record per procedure code per claim 
 
S 
String; up to 7 
characters 
Blank fields 
 
Pharmacy Data File (take home Rx) 
Element 
Order 
Field Name 
Details 
Required or 
Situational 
Format/Field 
Length 
Error Causes 
A 
Study ID 
Unique MNCARES assigned non-PHI study ID for 
patient 
R 
String; up to 
10 characters 
Blank fields 
B 
Claim ID 
 
R 
 
Blank fields 
C 
National Drug Code 
 
R 
String; 11 
digits 
Blank fields 
D 
Drug Name 
 
R 
 
Blank fields 
E 
Fill Date 
 
R 
mm/dd/yyyy 
or 
m/d/yyyy 
Blank fields 
F 
Units 
 
R 
String; up to 7 
characters 
Blank fields 
G 
Billed Amount 
This will be used only for calculation of Relative Value 
R 
Number, 2 
Digits passed 
decimal 
 
 
 

Appendix D 
MNCARES Organizational survey  
 
Version 1.2, May 1, 2023 
86 
17.4 Appendix D: Organizational Survey  
Thank you for taking the time to complete the following questions that will be used as part of the 
MNCARES research project. Together with clinic-level data, this information will be used to understand 
your approaches to care coordination for primary care patients and identifying other factors that might 
contribute to its effectiveness.  
 
We realize that there may have been large changes recently due to COVID-19 or other factors, so please 
answer the questions as your medical group exists today. We have also included questions designed 
specifically to understand how the COVID-19 pandemic may have affected care coordination within your 
medical group, as well as future planned changes.  
 
Please answer the questions as accurately as you can. If you are unsure of an answer, please provide an 
informed estimate. If you have any questions for us as you complete this, please contact [MNCARES 
Mailbox]. 
 
Please select which medical group you are affiliated with [medical group] *required 
 
___ [drop down list with all medical group names] 
 
Which of the following best describes your role within your medical group? [role] 
___Clinician leader 
___Care coordination leader 
___Administrative leader (non-care coordinator) 
___Other role, please specify: _______________________________ [role_other] 
 
A: The following questions ask about your medical group’s clinics and clinicians as of [date 
survey starts]. 
 
How many primary care clinics are there in your medical group? [cliniccount] 
________  
 
 
How many clinics in your medical group are reported separately to MN Community Measurement? 
[clinicreport] 
________  
 
How many clinics within your medical group are Health Care Home (HCH)-certified? [clinicHCH] 
_______  
 
How many hospitals are part of your medical group? [hospitalcount] 
_______  
 
Approximately how many adult primary care clinicians who provide patient care at least ½ time are 
there in your medical group? [adultPCP] 
_______  
 
Of these, how many are MD/DOs? [adultPCP_MD] ____ 
How many are NP/PA advanced practice clinicians? [adultPCP_APC] ____ 

Appendix D 
MNCARES Organizational survey  
 
Version 1.2, May 1, 2023 
87 
 
Approximately how many other clinicians who provide non-primary care specialty services are there in 
your medical group? For example, specialty physicians, dental professionals, NP or PA, etc. 
[othercinician] 
________ 
 
B: The following questions focus on how your medical group approaches care coordination for 
primary care patients at the current time. 
 
Across the primary care clinics in your medical group, approximately how many people are in the role of 
care coordinator [CCcount]? 
________  
 
Approximately how many primary care patients in your medical group are currently receiving care 
coordination services? [Patientcount] 
________  
 
In your medical group, are there patients who are not enrolled in the care coordination program but 
receive care coordination services? (short-term, partially, not fully enrolled) [partialCCenrollment] 
____ yes 
____ no 
 
In your medical group how do you define when a patient is fully enrolled in the care coordination 
program? Please list requirements for full enrollment in care coordination  [CCenrollment] 
______________________________________________________________________________
______________________________________________________________________________ 
 
On average, approximately how many patients in your medical group begin receiving care coordination 
services each month? (Please provide your best guess if a precise number is not known.) [Newpatients] 
_______  
 
What is the usual patient case load for a 1.0 FTE care coordinator? [caseload] 
______  
 
Which of the following types of personnel are included on care coordination teams in your medical 
group? Please check all that apply. [personneltype] 
____ RNs 
____ LPNs/CMAs 
____ community health workers (CHW) 
____ social workers 
____ non-clinical staff 
____ other, please specify: _______________________________ [personneltype_other] 
 
What percentage of patients receiving care coordination are “enrolled” for ongoing care coordination 
services? As opposed to those who only have discrete or limited services (such as 1-2 
calls/visits/reviews/etc.)  [percentenrolled] 
_____ % 

Appendix D 
MNCARES Organizational survey  
 
Version 1.2, May 1, 2023 
88 
 
Does your medical group target any of the following types of patients for care coordination? (Please 
check all that are targeted). [patienttype] 
____ any patient that a clinician wants to have those services 
____ patients transitioning between acute, post-acute and ambulatory care 
____ patients that are particularly complex medically 
____ patients with social or community resource needs 
____ patients who might experience disparities by race, income, comorbidities, etc. 
____ other (please explain) ____________________________ [patienttype_other] 
 
Do care coordinators in your medical group provide any of the following kinds of services? (Please check 
all that apply). [services] 
____ assistance finding culturally appropriate resources 
____ assistance with accessing health insurance 
____ disease management services 
____ employment assistance and referrals 
____ facilitating medical/behavioral services by primary care providers 
____ facilitating medical/behavioral services by specialty providers 
____ financial needs assessment and referrals 
____ housing/transportation/food needs assessment and referrals 
____ mental health or emotional needs assessment and referrals 
____ patient education and health behavior counseling 
____ spiritual needs assessment and referrals 
____ transitional service needs (e.g. transitions in care or placement needs) 
____ referrals for other community resources  
____ other (please explain) _________________________ [services_other]  
 
Is any “tiering” or complexity tool used to assess the level of care coordination needed for individual 
patients? [tiering] 
____ yes, for all patients 
____ yes, but only for some patients 
____ no 
 
 
How many of the certified HCH clinics in your medical group currently have a social worker with 
dedicated time for care coordination activities at that clinic? By social worker we mean someone with an 
educational degree in social work. This individual does not need to be a licensed social worker. [clinicSW] 
_______  
 
C: Please answer the following questions about social workers with dedicated time for care 
coordination activities: 
 
[If more than one clinic in the organization clincSW >1], On average, how many clinics is 
each social worker assigned to? [SWcount] ___________ 
 
[If one or more >=1 here and for rest of section], Does the social worker normally interact with 
individual patients to provide them with care coordination services? [SWpatient] 

Appendix D 
MNCARES Organizational survey  
 
Version 1.2, May 1, 2023 
89 
____ yes 
____ no 
 
Does the social worker normally interact with individual clinicians about their individual patients? 
[SWclinician] 
____ yes  If yes, are these interactions regularly scheduled or ad hoc? 
[swclinician_yes] 
____ no, social worker only provides general information about community resources or 
how to handle specific types of patient problems  
     
 
Does the social worker normally work on site at each assigned clinic at least 1 day/week? [SWonsite] 
____ yes, for all assigned clinics 
____ for some, but not all assigned clinics 
____  no 
 
Does your medical group bill any of the following payment sources for care coordination services? 
(Please check all that apply). [paymentsources] 
____ Medicare 
____ Medicaid/Medical Assistance/PMAP/MinnesotaCare 
____ commercial insurance 
____ specific ACO encounters/visits 
____ patients (without insurance or out-of-pocket)  
____ none 
____ other (please explain) ___________________________ [paymentsources_other] 
 
Does your medical group participate in any Value Based/Risk Based contracts or Accountable Care 
Organization agreements that include financial incentives for care coordination specifically? [ValueACO] 
____ Yes  If yes, (what % of your care coordination patients are in this category? 
[ValueACOper] _______% 
____ No 
 
D: Because we will be studying some patients who began receiving care coordination services in 
2018, we want to know whether any aspect of the way your medical group currently provides 
care coordination was different in 2018. (In case you have trouble recalling this time period, 
January 2018 was when Jan Malcom was appointed the commissioner of the Minnesota 
Department of Health).   
 
Previously you told us that your medical group currently includes the following types of personnel on 
care coordination teams: [Pipe in response values from question above: Which of the following types of 
personnel are included on care coordination teams in your medical group?]. Was this different in 2018? 
[personneldiff_2018] 
__Yes  Which of the following types of personnel were included on care coordination 
teams in 2018? (check all that apply, use same response options as red question above] 
[personneltype_2018] 
 
 
__No  
 

Appendix D 
MNCARES Organizational survey  
 
Version 1.2, May 1, 2023 
90 
[If social worker in 2018], In 2018, did the social worker normally interact with individual patients to 
provide them with care coordination services? [SWpatient_2018] 
____ yes 
____ no 
 
[If social worker in 2018], In 2018, did the social worker normally interact with clinicians about their 
individual patients? [SWclinician_2018] 
____ yes    If yes, were these interactions regularly scheduled or ad hoc? 
[swclinician_2018_yes] 
____ no, only provided general information about community resources or how to  
          handle specific types of patient problems 
 
[If social worker in 2018], In 2018, did the social worker normally work on site at each assigned clinic 
at least 1 day/week? [SWonsite_2018] 
____ yes, for all assigned clinics 
____ for some, but not all assigned clinics 
____  no 
 
Were there any other differences in the approach to care coordination in 2018 as compared to the 
current description in Sections B and C above? Please describe any other differences: (e.g. how patients 
were enrolled, types of patients served, services provided, tools used to assess patient complexity, 
number of care coordinators, interactions with patients and/or care team, payment or anything else.) 
[Approach2018] 
______________________________________________________________________________
______________________________________________________________________________ 
 
E. We are interested in any major changes in approach to care coordination that may have 
occurred within your medical group since the onset of the COVID-19 pandemic in March 2020 
or are planned for the future.  
 
Previously you told us that your medical group currently includes the following types of personnel on 
care coordination teams: [Pipe in values from red question above]. Has this changed since the onset of 
the COVID-19 Pandemic? [personnel_c19] 
__Yes  Which of the following types of personnel were included on care coordination 
teams prior to the onset of the COVID-19 Pandemic? (check all that apply, use same 
response options as red question above] [personneltype_c19] 
 
 
__No  
 
[If social worker included prior to COVID], Prior to the onset of the COVID-19 Pandemic, did the 
social worker normally interact with individual patients to provide care coordination services? 
[SWpatient_c19] 
____ yes 
____ no 
 
[If social worker included prior to COVID], Prior to the onset of the COVID-19 Pandemic, did the 
social worker normally interact with clinicians about individual patients? [SWclinician_c19] 

Appendix D 
MNCARES Organizational survey  
 
Version 1.2, May 1, 2023 
91 
____ yes    If yes, were these interactions regularly scheduled or ad hoc? 
____ no, only provided general information about community resources or how to  
          handle specific types of patient problems 
 
[If social worker included prior to COVID], Prior to the onset of the COVID-19 Pandemic, did the 
social worker normally work on site at each assigned clinic at least 1 day/week? [SWonsite_c19] 
____ yes, for all assigned clinics 
____ for some, but not all assigned clinics 
____  no 
 
Are there any other differences in the approach to care coordination since the onset of the COVID-19 
pandemic as compared to the current description in Sections B and C above? Please describe any other 
differences: (e.g. how patients were enrolled, types of patients served, services provided, tools used to 
assess patient complexity, number of care coordinators, interactions with patients and/or care team, 
payment or anything else.) [ApproachCOVID] 
______________________________________________________________________________
______________________________________________________________________________ 
 
Has your medical group needed to reduce the budget for care coordination since the onset of the 
COVID-19 pandemic? [CCCovidbudget] 
___ yes, considerably 
___ yes, somewhat 
___ no 
 
Are any other major changes planned for the next year in how care coordination will be implemented in 
your medical group? If yes, please describe. [CCfuture] 
______________________________________________________________________________
______________________________________________________________________________ 
 
F. Now we have a few final questions about your thoughts on what makes care coordination 
work well. 
 
How does your medical group measure whether your coordination program is a success? [measure] 
_____________________________________________________ 
 
 
What do you think are the main barriers or challenges to the effectiveness of care coordination within 
your medical group? [barrierchallenge] 
______________________________________________________________________________
______________________________________________________________________________ 
 
What factors or strategies help make care coordination in your medical group effective? [effective] 
______________________________________________________________________________
______________________________________________________________________________ 
 
What other factors or strategies not currently being used would make care coordination in your medical 
group more effective? [strategies] 

Appendix D 
MNCARES Organizational survey  
 
Version 1.2, May 1, 2023 
92 
______________________________________________________________________________
______________________________________________________________________________ 
 
What questions do you have about care coordination that could be answered by this study or other 
related research? [ccquestions] 
______________________________________________________________________________
______________________________________________________________________________ 
 
From the perspective of your medical group, what are the main benefits of providing care coordination? 
[MGbenefit] 
______________________________________________________________________________
______________________________________________________________________________ 
 
From the patient perspective, what does your medical group think the main benefits of receiving care 
coordination are for patients? [PatientBenefit] 
______________________________________________________________________________
______________________________________________________________________________ 
 
Please tell us about any policies and/or plans related to care coordination that your medical group has 
to measure and/or address disparities in healthcare. [Disparities] 
______________________________________________________________________________
______________________________________________________________________________ 
 
 
Thank you for your responses. You may submit by clicking the button below. Please contact us at 
[MNCARES Mailbox] if you have any questions about this form or the additional clinic level data that you 
or a designee is providing about your medical group pursuant to earlier conversations with the 
MNCARES study team. 
 
 
 

Appendix E 
MNCARES Clinic Descriptors Table V1.0 
 
Version 1.2, May 1, 2023 
93 
17.5 Appendix E: Clinic descriptors table  
MNCARES Organizational Survey Clinic Descriptors 
We are interested in knowing more about each clinic in your medical group that is certified as a Health Care 
Home. Please have someone from your team complete the following table for each clinic.  
Study_Org_ID   
MNCM_Org_ID   
MNCM_Clinic_ID 
  
Clinic Name 
Part 1: Clinic Operations 
1. Which of the following best describes this clinic’s ownership? (Place an X in all that apply) 
  
1a. Clinician-owned 
  
1b. Hospital/Health system owned 
  
1c. Federally Qualified Health Center or Look-Alike 
  
1d. Non-federal government clinic (state, county, city, public health clinic, etc.) 
  
1e. Residency training clinic  
  
1f. Federal (Military,  Veterans Administration,  Department of Defense) 
  
1g. Rural Health Clinic 
  
1h. Indian Health Service 
  
1i. Other 
  
Describe the "other" ownership: 
  
2. For the following role types, only include people who work at least 1/2 time at the clinic 
(Provide the number) 
  
2a. Approximately how many adult primary care physicians (MD/DO) are in this clinic? 
  
2b. Approximately how many adult primary care nurse practitioners are in this clinic? 
  
2c. Approximately how many adult primary care physician assistants are in this clinic? 
  
2d. Approximately how many behavioral health clinicians are in this clinic? 
  
2e. Approximately how many medical specialist MDs (excluding behavioral health) are in this 
clinic? 
  
2g. Approximately how many total other staff (including receptionists, CMA/LPN, lab techs, etc) 
are in this clinic? 
  
3. Does this clinic have an on-site pharmacy? (Yes or No) 
  
Part 2: Patient Demographics 
  
4. Insurance type 
  
4a. Approximately what percent of the patients seen in this clinic have Commercial Insurance? 
  
4b. Approximately what percent of the patients seen in this clinic have Medicare insurance? 
  
4c. Approximately what percent of the patients seen in this clinic have Medicaid insurance? 
  
4d. Approximately what percent of the patients seen in this clinic are uninsured?  
  
4e. Other not listed above  
  

Appendix E 
MNCARES Clinic Descriptors Table V1.0 
 
Version 1.2, May 1, 2023 
94 
Add up to a total of 100%   
5. Age  
5a. Approximately what percent of patients seen in this clinic are 18-39 years old? 
  
5b. Approximately what percent of patients seen in this clinic are 40-64 years old? 
  
5c. Approximately what percent of patients seen in this clinic are 65 years or older? 
  
Add up to a total of 100%   
6. Race   
6a. Approximately what percent of patients seen in this clinic are American Indian? 
  
6b. Approximately what percent of patients seen in this clinic are Asian? 
  
6c. Approximately what percent of patients seen in this clinic are Black/African American? 
  
6d. Approximately what percent of patients seen in this clinic are Hawaiian/Pacific Islander? 
  
6e. Approximately what percent of patients seen in this clinic are White? 
  
6f. Approximately what percent of patients seen in this clinic are some other race, multi-racial, 
or unknown? 
  
Add up to a total of 100%   
7. Ethnicity  
7a. Approximately what percent of patients seen in this clinic are Hispanic or Latino? 
  
7b. Approximately what percent of patients seen in this clinic are not Hispanic or Latino? 
  
7c. Approximately what percent of patients seen in this clinic have an unknown ethnicity? 
  
Add up to a total of 100%   
8. Language  
8a. Approximately what percent of patients seen in this clinic speak English? 
  
8b. Approximately what percent of patients seen in this clinic do not speak English? 
  
Add up to a total of 100%   
8c. Approximately what percent of patients seen in this clinic need an interpreter? 
  

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
95 
17.6 Appendix F: Care coordinator survey 
We thank your organization, its leaders, and you for participating in the MN Care Coordination 
Effectiveness Study (MNCARES). 
 
This survey is critical to our ability to understand what approaches to care coordination lead to the best 
outcomes for patients. We will share these learnings with your health system and others across the 
state. Your answers to this questionnaire are the only information we will have for this study to describe 
care coordination for this clinic, so please do your best to answer all the questions. If needed, please feel 
free to talk with your colleagues about any questions where you are not sure of the answer. We believe 
this survey will take about 20 minutes. Your responses will not be associated with you individually and 
will be combined with those from other care coordinators across the state and reported in such a way 
that you and your clinic will not be identifiable.  
 
Note: For best user experience, please avoid using Internet Explorer to complete this survey. 
 
****** 
 
We are interested in care coordination for adults who have a wide variety of problems. That is, we are 
not interested in care coordination that focuses on children or specific problems like drug abuse, or 
other specialized types of patients.  
 
1. Including yourself, how many care coordinators serve [clinic name] in total? [cc_num] __________ 
 
2. How many hours per week are devoted to care coordination in total at [clinic name] by care 
coordinators for adult patients with a wide variety of problems at [clinic name]?  [hrs_wk_dedicated] 
6, More than 100 hours 
5, 81-100 hours 
4, 61-80 hours 
3, 41-60 hours 
 
2, 20-40 hours 
 
1, Less than 20 hours  
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
3. During 2021, was dedicated time for care coordination by care coordinators at your clinic reduced for 
≥3 months for any reason? [covid_reduction] 
               0, No 
               1, Yes, Up to 25% reduction in care coordination time 
               2, Yes, 26-50% reduction 
               3, Yes,51-75% reduction 
               4, Yes, 76-100% reduction 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
4. Which of the following degrees do you have? Please check all that apply. [degrees] 
1, Registered Nurse  
2, Master of Nursing (MSN)  

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
96 
3, Licensed Practical Nurse (LPN)  
4, Certified Medical Assistant (CMA)  
5, Bachelor of Social Work (BSW/BSSW) 
6, Master of Social Work (MSW) 
7, Doctor of Social Work (DSW)  
8, Master of Public Health (MPH) 
9, Associate in Arts (AA) 
10, Bachelor of Arts (BA) 
11, Bachelor of Science (BS) 
99, Other, please specify: ___________ [degrees_oth] 
 
5. How many years have you worked, either physically or remotely from another location, as a care 
coordinator at [clinic name]? [yrs_clinic] 
 
(text box) 
 
6. How many years have you worked as a care coordinator in total, including [clinic name] and others? 
[yrs_cc] 
 
(validation with error box q5 answer>q4 answer)(text box) 
 
7. How many hours per week do you usually spend doing care coordination work at [clinic name] 
(including in-person and remote)? ___ [require integer – max value = 60] [hrs_week] 
 
 [if reporting for more than 1 clinic] Ask question for each specific clinic 
 
8. In an average week, how often are you physically in [clinic name], as opposed to working remotely or 
at another site? [physically_clinic] 
 
3, Always physically at this clinic 
 
2, Sometimes at this clinic 
 
1, Rarely or never at this clinic 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
9. Are you certified in care coordination? [cc_certif] 
 
1, Yes  
0, No 
 
10. Does [clinic name] have any of the following to support care coordination work?   
Dedicated space to meet with patients face-to-face [space] 
 
 
Yes, 1 No, 0  
Electronic Medical Record (EMR) monitoring functions and prompts  
(information in the EMR to remind or prompt that a particular patient needs  
follow-up, test or treatment change)  
[ehr] 
 
 
 
 
Yes 
No  
Care coordination patient registry (paper or electronic list of  
patients in care coordination that can be searched to identify  
those who need follow-up) 
[registry] 
 
 
 
 
Yes 
No 
Support for coordination from admin staff (non-clinical personnel  
who can assist with care coordination services, for example making  
appointments or contacting care coordination patients). [staff]  
 
Yes 
No 
 

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
97 
[if at more than 1 clinic] Ask question for each specific clinic 
 
[Page break] 
 
The following questions are about social workers that may be part of your care team. Please answer as 
best as you can. If you are not sure of an answer, feel free to reach out to your colleagues or provide 
your best guess. For some of these questions we will ask about how things were before the onset of the 
COVID-19 pandemic in about March 2020 as well as any changes planned this year. 
 
11. Is there at least one person with a social worker degree (licensed or not) who is considered part of 
the care team at [clinic name] and works on site at least some of the time? [sw_team] 
1, Yes  
0, No  
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
[If Yes to Q11]11a. Is the social worker a care coordinator (that is, do they have a personal panel 
of adult care coordination patients they are assigned to provide coordination services to over 
time)? [sw_cc] 
 
1, Yes 
 
0, No 
 
[If Yes to Q11] 11b. How many social workers are considered part of the care team at [clinic 
name]? [sw_team_num] 
 
1, 1 
 
2, 2 
 
3, 3 or more 
 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
[If Yes to Q11] 11c. Are they responsible for assessing and coordinating social services for care 
coordination patients at [clinic name]? [sw_responsible] 
 
2, Yes, for all patients in care coordination 
1, Yes, but only for those patients that are referred to the social worker because of need 
for social services 
 
0, No 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
[If Yes to Q11] 11d. Do they regularly interact with care coordination patients? [sw_interact] 
 
1, Yes 
 
0, No 
 
[If yes, interact w patients] 11da. Who usually initiates interactions between the social 
worker and the patient? [sw_initiate] 

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
98 
 
 
 
1, Most often the social worker 
 
2, About equal 
 
3, Most often the patient 
 
[If yes, interact w patients] 11db. About how often are each of the following forms of 
communication used for the social worker and patient interaction? [sw_com] 
 
 
[Always used, mostly used, sometimes used, rarely used, never used] 
 
 
 
4 
 
3 
2 
 
1 
 
0 
In-person meeting [sw_com_meet] 
Telephone [sw_com_phone] 
Video call [sw_com_video] 
Message through EMR [sw_com_emr] 
Some other, [sw_com_oth] Please specify: ________ [sw_com_other] 
 
[If yes, interact w patients] 11dc. On average, how many interactions does the social 
worker have each month with patients receiving ongoing care coordination? 
[sw_interact_num] 
 
0, Less than once per month 
1, 1 
2, 2 
3, 3 
4, 4 or more 
 
 
[If Yes to Q11] 11e. Does the social worker routinely interact with clinicians of care coordinated 
patients regarding their care? [sw_interact_clinician] 
1, Yes 
 
0, No 
 
[If yes, interact w clinicians] 11ee. Which of the following forms of communication are 
used for the social worker and clinician interaction at [clinic name]? Check all that apply. 
[sw_communicate_clin] 
 
1, In-person meeting 
2, Ad hoc in-person conversation 
3, Telephone 
4, Video call 
5, Message through EMR 
99, Some other, Please specify: ________ [sw_communicate_oth] 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
[If Yes to Q11] 11f. How easy or difficult is it for you to engage help from the social worker when 
needed? [sw_engage] 
 
4, Very easy 
 
3, Somewhat easy 
 
2, Somewhat difficult 
 
1, Very difficult 

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
99 
99, N/A (I am the social worker) 
[Page break] 
 
12. Before the onset of the COVID-19 pandemic was there at least one person with a social worker 
degree (licensed or not) who was considered part of the care team at [clinic name] and worked on site 
at least some of the time? [before_covid_sw] 
 
1, Yes 
 
0, No 
 
99, Not sure 
 
[if reporting for  more than 1 clinic] Ask question for each specific clinic 
 
[If yes to Q12] 12a. Did they regularly interact with care coordination patients? 
[covid_sw_interact_pat] 
 
 
1, Yes 
 
 
0, No 
 
 
99, Not sure 
 
[if reporting for  more than 1 clinic] Ask question for each specific clinic 
 
[If yes to Q12] 12b. Did they routinely interact with clinicians of care coordinated 
patients regarding their care? [covid_sw_interact_clin] 
1, Yes 
 
0, No 
 
99, Not sure 
 
[if reporting for  more than 1 clinic] Ask question for each specific clinic 
 
[if yes to Q12] 12c. Does [clinic name] plan to continue to have a social worker (licensed 
or not) that is on site at least some of the time and part of the care team for the 
remainder of this year? [covid_sw_plan] 
1, Yes 
 
0, No 
 
99, Not sure 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
[If yes] 12ca. Would they regularly interact with care coordination 
patients?  [plan_interact_pat] 
 
 
1, Yes 
 
 
0, No 
 
 
99, Not sure 
 
 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
[If yes] 12cb. Would they routinely interact with clinicians of care 
coordinated patients regarding their care? [plan_interact_clin] 

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
100 
1, Yes 
 
 
0, No 
 
 
99, Not sure 
 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
[if no or not sure to Q12] 13a. Is there any plan to add a person with a social worker 
degree (licensed or not) who would be on site at least some of the time to the care 
team at [clinic name] yet this year? [plan_add_sw] 
1, Yes 
 
 
0, No 
 
99, Not sure 
 
[if at more than 1 clinic] Ask question for each specific clinic 
 
[If yes]12aa.  Will they regularly interact with care coordination 
patients? [plan_add_sw_pat] 
 
 
1, Yes 
 
 
0, No 
 
 
99, Not sure 
 
[if at more than 1 clinic] Ask question for each specific clinic 
 
[If yes] 12ab. Will they routinely interact with clinicians of care 
coordinated patients regarding their care? [plan_add_sw_clin] 
1, Yes 
 
0, No 
 
99, Not sure 
 
[if at more than 1 clinic] Ask question for each specific clinic 
 
[Page break] 
 
Now thinking about care coordination generally. 
 
13. On average how many patients does a care coordinator work with at any given time at [clinic name]? 
[patients_num] 
 
___ [require integer] 
[if at more than 1 clinic] Ask question for each specific clinic 
 
14. Does this number of patients seem like too many, about right or too few? [patients_num_load] 
 
Too many 
 
 
About right 
 
Too few 
[if at more than 1 clinic] Ask question for each specific clinic 
 

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
101 
15. On average, how many clinicians does a full-time care coordinator generally work with at any given 
time? [clinicians_to_cc] 
 
___ [require integer] 
 
16. Generally, how are patients that need care coordination identified? Please check all that apply. 
[identification] 
 
1, Referral from clinician 
 
2, Referral from other staff 
 
3, High utilizers of hospital/Emergency Department care 
 
4, Particular health conditions (e.g., diabetes, drug addiction, etc.) 
 
5, Transition in site of care (e.g., inpatient discharged to home) 
6, Patients identified through registry 
7, Patients with a complex condition or multiple co-morbidities 
 
8, High social needs 
 
9, Patient/family request 
 
99, Some other way, please specify: _____________ [identification_oth] 
 
 
17. Generally, about how often are each of the following forms of communication used for interactions 
between care coordinators and care coordination patients?  
 
 
[Always used, mostly used, sometimes used, rarely used, never used] 
 
 
 
4 
3 
 
2 
 
1 
 
0 
In-person meeting [communication_meet] 
Telephone [communication_phone] 
Video call [communication_video] 
Message through EMR [communication_emr] 
Some other, [communication_oth] Please specify: ________ [communication_other] 
 
18. Who usually initiates interactions with care coordination patients? [initiate_interaction] 
 
1, Most often the care coordinator initiates the interaction 
 
2, About equal 
 
3, Most often the patient initiates the interaction 
 
19. How often do care coordinators at [clinic name] usually engage with the family and/or caregiver(s) of 
care coordination patients? [engage_family] 
 
3, Regularly/often 
 
2, Sometimes 
 
1, Rarely 
 
0, Never 
 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
20. Are care coordinators at [clinic name] considered part of the primary care team? [cc_team] 
 
2, Yes, definitely 
 
1, Yes, somewhat 
 
0, No 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
102 
21. Generally, how often do care coordinators interact with clinicians of patients receiving care 
coordination before they interact with the patients? [interact_clin_before] 
3, Regularly/often 
 
2, Sometimes 
 
1, Rarely 
 
0, Never 
 
22. Generally, how often do care coordinators interact with clinicians of patients receiving care 
coordination after they interact with the patient? [interact_clin_after] 
3, Regularly/often 
 
2, Sometimes 
 
1, Rarely 
 
0, Never 
 
23. Generally, how often are each of the following forms of communication used for interactions 
between care coordinators and the primary care team? 
 
 
[Always used, mostly used, sometimes used, rarely used, never used] 
 
 
 
4 
3 
 
2 
 
1 
 
0 
In-person meeting [com_team_meet] 
Ad hoc in-person conversation [com_team_convo] 
Telephone [com_team_phone] 
Video call [com_team_call] 
Message through EMR [com_team_emr] 
Some other, [com_team_oth] Please specify: ________ [com_team_other] 
 
[Page break] 
 
24. To what extent do you agree or disagree with the following statements: As a reminder your results 
will only be reported in aggregate with responses from other care coordinators across the state. 
 
a. Most clinicians at [clinic name] value the care coordinator role. [clinician_value] 
4, Strongly agree 
3, Somewhat agree 
2, Somewhat disagree 
1, Strongly disagree 
 
b. Clinic leadership at [clinic name] values the care coordinator role. [leadership_value] 
4, Strongly agree 
3, Somewhat agree 
2, Somewhat disagree 
 
1, Strongly disagree 
 
[if reporting fort more than 1 clinic] Ask question for each specific clinic 
 
25. What other resources within your organization are readily available to the care team at [clinic 
name]? Please check all that apply. [rsrc_avail] 
1, Community Health Workers (CHWs) 

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
103 
2, Pharmacist(s) 
3, Behavioral Health Services 
4, Other medical specialties 
5, Other surgical specialties 
99, Some other, Please specify: ________ [rsrc_avail_oth] 
[if reporting fort more than 1 clinic] Ask question for each specific clinic 
 
26. Generally, how often is payment or coverage for Care Coordination services required to provide care 
coordination services for a specific patient? [coverage] 
 
2, Most of the time 
 
1, Sometimes 
 
0, Rarely or never 
 
 
27. What Care Coordination services are provided at [clinic name]? Please check all that apply. 
[cc_services] 
1, Assistance finding culturally appropriate resources 
2, Assistance with accessing health insurance 
3, Disease management services 
4, Employment assistance and referrals 
5, Facilitating medical/behavioral services by primary care providers 
6, Facilitating medical/behavioral services by specialty providers 
7, Financial needs assessment and referrals 
8, Housing/transportation/food needs assessment and referrals 
9, Mental health or emotional needs assessment and referrals 
10, Patient education and health behavior counseling 
11, Spiritual needs assessment and referrals 
12, Transitional service needs (e.g. transitions in care or placement needs) 
13, Referrals for other community resources 
99, Other, please specify: ________ [cc_services_oth] 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 
 
28. How familiar are you with the clinical resources available in your organization (e.g., various 
specialties, mental health care, PT/OT)? [familiar_clinic_rsrc] 
2, Very familiar 
1, Somewhat familiar 
0, Not at all familiar 
 
29. Do you have established personal relationships with individuals within these clinical resources? 
[relat_clin_rsrcs] 
 
2, Yes, many 
 
1, Yes, some 
 
0, No 
 
30. How familiar are you with the community resources available in your area? [familiar_com_rsrcs] 
2, Very familiar 
1, Somewhat familiar 
0, Not at all familiar 

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
104 
 
31. Do you have established personal relationships with individuals within these community resources? 
[relat_com_rsrcs] 
 
2, Yes, many 
 
1, Yes, some 
 
0, No 
 
32. How do you help connect patients with resources? Please check all that apply. [connect_rsrcs] 
1, Give the patient a name and phone number 
2, Contact the resource with a patient referral 
3, Call referred resource with the patient or help with necessary applications 
99, Some other way, Please specify: __________ [connect_rsrcs_oth] 
 
[page break] 
 
33. To what extent are care coordinators usually involved in facilitating care transitions (for example, 
home to hospital or hospital to home) for care coordination patients? [facilitate_transitions] 
 
2, Very involved 
1, Somewhat involved 
0, Not at all involved  
 
 
34. Overall, what proportion of newly enrolled care coordination patients are assessed for complexity of 
their medical needs using a standardized format or instrument? [standard_assess_med] 
4, All 
3, Most 
2, About half 
1, Some 
0, None 
 
35. Overall, what proportion of newly enrolled care coordination patients are assessed for complexity of 
their social needs using a standardized format or instrument? [standard_assess_social] 
4, All 
3, Most 
2, About half 
1, Some 
0, None 
 
36. How often do care coordinators at [clinic name] engage in each of the following for care 
coordination patients with specific needs? 
 
2, Most of the time 
 
1, Some of the time 
 
0, Rarely or never 
 
 
 
 
Refer to services outside your care system/organization [engage_refer_out] 
 
 
Refer to services within your care system/organization [engage_refer_in] 
 
 
Directly provide services [engage_provide] 
 
[if reporting for more than 1 clinic] Ask question for each specific clinic 

Appendix F 
MNCARES Care coordinator survey 
Version 1.2, May 1, 2023 
105 
 
37. What measures does your clinic/care system use to regularly evaluate the effectiveness of its care 
coordination services? Please select all that apply. [measures] 
 
1, Types of patients seen   
2, Types of services provided  
3, Utilization of hospital/Emergency Department services   
4, Volume of patients seen by care coordinators 
5, Changes in hospitalizations or emergency department visits  
6, Changes in the control of chronic conditions  
7, Measures of care coordination patient satisfaction 
 
99, Other (describe) [measures_oth] 
 
66, We don’t monitor any data for just these patients  
 
38. How often are specialty medical services needed by your care coordination patients readily available 
to them? [med_available] 
 
3, Most of the time 
 
2, Some of the time 
 
1, Rarely 
 
390. How often are community services needed by your care coordination patients readily available to 
them in your area? [com_available] 
3, Most of the time 
 
2, Some of the time 
 
1, Rarely  
 
 
40. How easy or difficult is it to coordinate services with the hospitals or nursing homes used by your 
patients? [coord_hosp] 
4, Very easy 
3, Somewhat easy 
2, Somewhat difficult 
1, Very difficult 
 
41. How often do patient financial constraints limit their access to needed social and medical services? 
[financial_limit] 
3, Most of the time 
 
2, Some of the time 
 
1, Rarely  
 
 
42. Overall, how satisfied are you with the time and resources that you have currently to provide 
needed care coordination services? [satisfaction] 
4, Very satisfied 
3, Somewhat satisfied 
2, Somewhat dissatisfied 
1, Very dissatisfied 
 
Thank you for taking the time to respond to this survey and give us feedback. Your responses are 
important.

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
106 
17.7 Appendix G: Historical Cohort patient survey 
MNCARES – Perspectives on your health and care  
 
Thank you for taking the time to answer these questions about your experiences with health and health 
care! You were identified for this survey by your clinic because you may have received care coordination 
starting in 2018. We are trying to learn how to improve those important services in order to achieve the 
best results for patients’ health.  
We are interested in knowing more about your health and health care experiences. We are also 
interested in how these things have changed (or not changed) as a result of COVID-19 and the 
disruptions that followed.  
 
There are no right or wrong answers. Your experiences are valuable and will help to answer some very 
important questions about care coordination services during an unprecedented time. There is space at 
the end of the survey for you to tell us more if you wish. This could include more about your responses 
or anything else you would like to share about your experiences with health and health care now or how 
things may have changed as a result of the COVID-19 pandemic and the disruptions that followed. 
 
1. How would you rate your health in general now? (BRFSS, NHIS, MEPS have similar questions) 
Excellent 
Very good 
Good 
Fair  
Poor 
 
2. How has your health changed as a result of the COVID-19 pandemic? Would you say your health 
is…?  
Much worse 
Somewhat worse 
No change 
Somewhat better 
Much better 
 
3. On a scale from 0-10 where 0 means very dissatisfied and 10 means very satisfied, how do you feel 
about your life as a whole these days? (NHIS and relative comparison in HWB) 
 
0 Very dissatisfied 
 
1 
 
2 
 
3 
 
4 
 
5 
 
6 
 
7 
 
8 
 
9 
 
10 Very satisfied 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
107 
 
4. How has your satisfaction with life as a whole changed as a result of the COVID-19 pandemic? 
Would you say it is…? 
Much worse 
Somewhat worse 
No change 
Somewhat better 
Much better 
 
5. How much do physical health concerns get in the way of your life now? (similar to NHIS) And HWB) 
 
Not at all 
A little bit 
Some 
Quite a bit 
Completely 
 
6. How have any physical health concerns changed as a result of the COVID-19 pandemic? Would you 
say your physical health is…? 
Much worse 
Somewhat worse 
No change 
Somewhat better 
Much better 
 
7. How much do emotional health concerns, such as feeling depressed, isolated or anxious, get in the 
way of your life now? (NHIS And HWB) 
Not at all 
A little bit 
Some 
Quite a bit 
Completely 
 
8. How have any emotional health concerns changed as a result of the COVID-19 pandemic? Would 
you say your emotional health is…? 
Much worse 
Somewhat worse 
No change 
Somewhat better 
Much better 
 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
108 
9. On a scale from 0 to 10, where 0 is the worst primary care clinic possible, and 10 is the best primary 
care clinic possible, what number would you use to rate your current primary care clinic? (CG 
CAHPS) 
 
0 Worst primary care clinic possible 
 
1 
 
2 
 
3 
 
4 
 
5 
 
6 
 
7 
 
8 
 
9 
 
10 Best primary care clinic possible 
10. How has your rating of your primary care clinic changed as a result of the COVID-19 pandemic? 
Would you say it is…? 
Much worse 
Somewhat worse 
No change 
Somewhat better 
Much better 
 
11. In general, when you contact your primary care clinic to get an appointment for care you need right 
away, how often do you get an appointment as soon as you need? (similar to CG CAHPS) 
Never 
Sometimes  
Usually  
Always 
Does not apply to me because I never need care right away 
 
12. How has your ability to get care you need right away changed as a result of the COVID-19 pandemic? 
Would you say it is…? 
Much worse 
Somewhat worse 
No change 
Somewhat better 
Much better 
Does not apply to me because I never need care right away 
 
 
13. Thinking about all the care you typically receive from your primary care clinic (including phone, 
clinic, video or home visits)... (Elwyn, CollaboRATE) 
 
a. How much effort is made to help you understand your health issues? 
No effort is made 
A little effort is made 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
109 
Some effort is made 
A lot of effort is made 
Every effort is made 
 
b. How much effort is made to listen to the things that matter most to you about your health 
issues? 
No effort is made 
A little effort is made 
Some effort is made 
A lot of effort is made 
Every effort is made 
 
c. How much effort is made to include what matters most to you in choosing what to do next? 
No effort is made 
A little effort is made 
Some effort is made 
A lot of effort is made 
Every effort is made 
 
14. Receiving health care often means seeing different people, such as office staff, nurses, doctors, and 
other health professionals. Please think about a health issue that leads you to see different health 
professionals and answer the following questions. (Elwyn, Integrate) 
 
a. How often do you have to do or explain something because people do not share information 
with each other? 
Never 
A little 
A lot 
Always 
 
b. How often are you confused because people give you conflicting information or advice? 
Never 
A little 
A lot 
Always 
 
c. How often do you feel uncomfortable because people do not get along with each other? 
Never 
A little 
A lot 
Always 
 
d. How often are you unclear whose job it is to deal with a specific question or concern? 
Never 
A little 
A lot 
Always 
 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
110 
15. How easy or difficult is it for you to understand information that doctors, nurses and other health 
professionals tell you? Would you say it is … (Modified BRFSS health literacy module) 
Very easy  
Somewhat easy  
Somewhat difficult  
Very difficult  
 
16. You can find written information about health on the Internet, in newspapers and magazines, and in 
brochures in the doctor’s office and clinic. In general, how easy or difficult is it for you to understand 
written health information? Would you say it is … (Modified BRFSS health literacy module) 
Very easy  
Somewhat easy  
Somewhat difficult  
Very difficult  
I don’t look for health information 
 
The next question is about care coordination. A care coordinator is a nurse or a social worker who helps 
patients manage their health or healthcare services. They can help with things like coordinating doctor's 
appointments or providing referrals for resources. This person might also be called a care manager, care 
navigator or health coach.  
Have you had any type of visit (including phone, clinic, video or home) with a care 
coordinator from your clinic in the past 6 months?” Yes 
No  Skip to Q19 
 
17. How often do you see or talk to a care coordinator these days? 
More than once a week 
About once a week 
A few times a month 
About once a month 
Less than once a month 
18. On a scale from 0 to 10, where 0 is the worst care coordinator possible, and 10 is the best care 
coordinator possible, what number would you use to rate your current care coordinator? (CAHPS, 
adaptation) 
 
0 Worst care coordinator possible 
 
1 
 
2 
 
3 
 
4 
 
5 
 
6 
 
7 
 
8 
 
9 
 
10 Best care coordinator possible 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
111 
19. At any time since the start of the COVID-19 pandemic, did you have a video visit with your doctor? 
[telemedicine] 
Yes 
No 
 
20. [If telemedicine = yes] How would you compare the value of a video visit with your doctor versus an 
office visit? 
Video visit is much more valuable than an office visit 
Video visit is somewhat more valuable 
Video visit and office visit are about the same 
Office visit is somewhat more valuable 
Office visit is much more valuable 
 
 
21. Please think about all of the things that you must do to take care of your health because of an 
ongoing illness or health condition, such as seeking and understanding medical information, taking 
medications, going to medical appointments, monitoring your health, diet, and exercise. (Burden of 
treatment, adaptation) 
On a scale of 0 to 10, with 0 being very easy and 10 being very difficult, over the past 4 weeks, how 
easy or difficult has it been for you to do all of the self-care tasks that you need to do?   
0 
1 
2 
3 
4 
5 
6 
7 
8 
 
9 
10 
Very 
easy 
 
 
 
 
Neither 
easy 
nor 
difficult 
 
 
 
 
 
Very 
difficult 
22. How has the ease or difficulty of your self-care tasks changed as a result of the COVID-19 pandemic? 
Would you say your self-care tasks are…?  
Much more difficult 
Somewhat more difficult 
No change 
Somewhat easier 
Much easier 
 
23. On a scale of 0 to 10, with 0 being not at all and 10 being very much, over the past 4 weeks, how 
much have these self-care tasks interfered with your everyday activities and your overall enjoyment 
of life? (Burden of treatment, adaptation) 
0 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
Not at 
all 
 
 
 
 
Somewhat 
 
 
 
 
Very 
much 
 
24. How has the interference of these self-care tasks on your everyday activities and your overall 
enjoyment of life changed as a result of the COVID-19 pandemic? Would you say it is…” 
Much worse 
Somewhat worse 
No change 
Somewhat better 
Much better 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
112 
 
Finally we have some questions about you and your living situation. These will help us better understand 
who answered this survey. 
25. Which of the following best describes your current living situation? (CMS HCSRN screener, The AHC 
Health-Related Social Needs Screening Tool (cms.gov)) 
I have a steady place to live 
I have a place to live today, but I am worried about losing it in the future 
I do not have a steady place to live (temporarily staying with others, in a hotel, in a shelter, living 
outside on the street, on a beach, in a car, abandoned building, bus or train station, or in a park) 
 
26. How has your living situation changed as a result of the COVID-19 pandemic? Would you say it is…?  
Much less steady 
Somewhat less steady 
No change 
Somewhat more steady 
Much more steady 
 
27. In the last 12 months, how often did you or other adults in your household ever eat less at a meal or 
skip meals because there wasn’t enough money for food? (CPS food security supplement) 
Often 
Sometimes 
Rarely 
Never 
 
28. How has the frequency that you or other adults in your household eat less or skip meals because 
there wasn’t enough money for food changed as a result of the COVID-19 pandemic? Would you say 
it occurs…?  
Much more often 
Somewhat more often 
No change 
Somewhat less often 
Much less often 
 
 
29. In the past 12 months, has lack of reliable transportation kept you from medical appointments, 
meetings, work or from getting things needed for daily living? (CMS HCSRN screener) 
Yes 
No 
 
30. How has the frequency that you had lack of reliable transportation changed as a result of the COVID-
19 pandemic? Would you say it occurs…?  
Much more often 
Somewhat more often 
No change 
Somewhat less often 
Much less often 
 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
113 
31. Do you need help with day-to-day activities such as bathing, preparing meals, shopping, managing 
finances, etc..? (CMS HCSRN screener, adapted) 
1, Yes 
0, No  Skip to question Q34 
 
32. If Q31 =1 [If need help] Do you get the help you need? (CMS HCSRN screener, adapted) 
Yes, definitely 
Yes, somewhat 
No 
 
33. If Q31 =1 [If need help] How has the frequency that you get the help that you need changed as a 
result of the COVID-19 pandemic? Would you say you get the help you need…?  
Much more often 
Somewhat more often 
No change 
Somewhat less often 
Much less often 
 
34. When you need advice or support are there family or friends you can turn to? (SMH/HWB) 
Yes 
No  
Don’t know 
 
35. How has the frequency that you get the advice or support you need from family and friends changed 
as a result of the COVID-19 pandemic? Would you say it occurs…?  
Much more often 
Somewhat more often 
No change 
Somewhat less often 
Much less often 
 
36. How often do you feel lonely or isolated from those around you? (CMS HCSRN screener) 
Never 
Rarely 
Sometimes 
Often  
Always 
 
37. How has the frequency that you feel lonely or isolated from those around you changed as a result of 
the COVID-19 pandemic? Would you say it occurs…?  
Much more often 
Somewhat more often 
No change 
Somewhat less often 
Much less often 
 
38. In the last 12 months, was there any time when you needed medical care, but did not get it because 
you couldn’t afford it? (MEPS) 
Yes 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
114 
No  
 
39. How has the frequency that you did not get needed medical care because you could not afford it 
changed as a result of the COVID-19 pandemic? Would you say it occurs…?  
Much more often 
Somewhat more often 
No change 
Somewhat less often 
Much less often 
 
40. In the last 12 months, was there any time when you needed prescription medicines, but did not get 
it because you couldn’t afford it? (MEPS) 
Yes 
No  
 
41. How has the frequency that you did not get needed prescription medicines because you could not 
afford it changed as a result of the COVID-19 pandemic? Would you say it occurs…?  
Much more often 
Somewhat more often 
No change 
Somewhat less often 
Much less often 
 
42. Approximately how much did anyone in your family pay ‘out-of-pocket’ (that is, including co-pays, 
coinsurance, and meeting of deductibles) for health care services you received in the last 12 
months? (Similar to Medicare CBS?) 
Zero 
Less than $500 
$500-$1,999 
$2,000-$2,999 
$3,000-$4,999 
$5,000 or more 
 
43. How has the amount that anyone in your family pay ‘out-of pocket’ for health care services you 
received changed as a result of the COVID-19 pandemic? Would you say it…?  
Increased a lot 
Increased a little 
No change 
Decreased a little 
Decreased a lot 
 
44. Do you currently have any kind of health care coverage, including health insurance, prepaid plans 
such as HMOs, or government plans such as Medicare, or Indian Health Service? (BRFSS) 
Yes 
No 
 
45. Did your health care coverage change as a result of the COVID-19 pandemic? 
No 
Yes, I lost health care coverage as a result of COVID-19 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
115 
Yes, I gained (additional or new) health care coverage as a result of COVID-19 
 
We have just a few more questions about you. 
 
46. Do you currently live alone? (could derive comparison from BRFSS or CAHPS) 
Yes 
No 
 
47. If Q26=0 [If do not live alone] How many members of your household, including yourself, are 18 or 
older? (could derive comparison from BRFSS) 
 
48.  
_ _ Number  
  
49. If Q26=0 [If do not live alone] How many members of your household are under 18? (could derive 
comparison from BRFSS) 
50.  
 _ _ Number  
 
51. What is your marital status? (BRFSS has similar question) 
Now married 
Widowed 
Divorced 
Separated 
Never married 
52. What is the highest grade or year of school you completed? (BRFSS) 
8th grade or less 
Some high school 
High school graduate 
Some college or technical school 
College graduate 
Some graduate school or more 
 
53. Are you currently….? (If more than one applies, select the category which best describes you.) 
(BRFSS) 
Employed for wages  
Self-employed  
Out of work for 1 year or more  
Out of work for less than 1 year  
A homemaker  
A student  
Retired 
Unable to work 
 
54. What is your best estimate of the total income of all family members living in your household from 
all sources before taxes in the last calendar year?  (BRFSS) 
Less than $15,000 
Between $15,000 and $29,999 
Between $30,000 and $49,999 

Appendix G 
MNCARES Historical Cohort patient survey 
Version 1.2, May 1, 2023 
116 
Between $50,000 and $99,999 
$100,000 or more 
 
55. How has your annual household income from all sources changed as a result of the COVID-19 
pandemic? Would you say it…?  
Increased a lot 
Increased a little 
No change 
Decreased a little 
Decreased a lot 
 
56. Did someone help you complete this survey? (CAHPS and similar question in CMS COVID 
questionnaire) 
Yes 
No 
 
57. Is there anything else that you would like to tell us about your responses or about your experiences 
with health and health care now or how things may have changed as a result of the COVID-19 
pandemic and the disruptions that followed? 
_________________________ 
_________________________ 
_________________________ 
 
 
To thank you for your time, we would like to send you a $10 gift card of your choice.  
- 
Select the gift card you would like to receive: 
o Target 
o Walmart 
- 
Please enter the best mailing address to send your gift card 
• 
Address 1: 
• 
Address 2: 
• 
City: 
• 
State: 
• 
Zipcode: 
 
Thank you for taking the time to complete this survey. Your responses are very important. 

Appendix H 
MNCARES Primary Cohort patient survey 
Version 1.2, May 1, 2023 
117 
17.8 Appendix H: Primary cohort patient survey 
MNCARES – Improving the coordination of primary care 
 
Thank you for taking the time to answer these questions about your experiences with health and health 
care! You were identified for this survey by your clinic because you may have received care coordination 
services. We are working with your clinic to learn how to improve the coordination of care for their 
patients. 
There are no right or wrong answers. Your experiences are valuable and will help to identify ways to 
improve care coordination services. There is space at the end of the survey for you to tell us more if you 
wish. This could include more about your responses or anything else you would like to share about your 
experiences with health and health care. 
[web survey] Note: For the best user experience, please avoid using Internet Explorer to complete this 
survey. 
[For phone, do not need to directly ask unless needed:] 
Before we begin, we would like to know who is completing this survey. Is it… [proxy] 
              1, The individual invited to complete the survey 
              2, A family member of the invited individual 
             3, A non-family caregiver of the invited individual 
              4, Both the individual invited to complete the survey and a family member or caregiver 
[If caregiver or family member responding to this survey on behalf of someone else] Please answer the 
questions as you think that person would respond. 
1. Please respond to each question or statement by marking one box per row. [phone only]: Read 
response options aloud for first row, then as needed for the rest of the matrix.  
5, Excellent 
4, Very good 
3, Good 
2, Fair 
1, Poor 
In general, would you say your health is: [global01] 
In general, would you say your quality of life is: [global02] 
In general, how would you rate your physical health? [global03] 
In general, how would you rate your mental health, including your mood and your ability to 
think? [global04] 
In general, how would you rate your satisfaction with your social activities and relationships? 
[global05] 
In general, please rate how well you carry out your usual social activities and roles. (This 
includes activities at home, at work and in your community, and responsibilities as a parent, 
child, spouse, employee, friend, etc.) [global09r] 
 
*Questions 1, 2, and 3 from PROMIS® Scale v1.2 – Global Health 
 

Appendix H 
MNCARES Primary Cohort patient survey 
Version 1.2, May 1, 2023 
118 
2. To what extent are you able to carry out your everyday physical activities such as walking, climbing 
stairs, carrying groceries, or moving a chair? [global06] 
5, Completely 
4, Mostly 
3, Moderately 
2, A little 
1, Not at all 
 
3. In the past 7 days…  
 
a. How often have you been bothered by emotional problems such as feeling anxious, depressed 
or irritable? [global10r] 
5, Never 
4, Rarely 
3, Sometimes 
2, Often 
1, Always 
b. How would you rate your fatigue on average? [global08r] 
5, None 
4, Mild 
3, Moderate 
2, Severe 
1, Very severe 
c. How would you rate your pain on average? [global07r] 
 
0 
No 
pain 
1 
2 
3 
4 
5 
6 
7 
8 
 
9 
10 
Worst 
pain 
imaginable 
*Questions 1, 2, and 3 from PROMIS® Scale v1.2 – Global Health 
 
4. On a scale from 0 to 10, where 0 is the worst primary care clinic possible, and 10 is the best primary 
care clinic possible, what number would you use to rate your current primary care clinic? [pc_rating] 
 
0 Worst primary care clinic possible 
 
1 
 
2 
 
3 
 
4 
 
5 
 
6 
 
7 
 
8 
 
9 
 
10 Best primary care clinic possible 
 

Appendix H 
MNCARES Primary Cohort patient survey 
Version 1.2, May 1, 2023 
119 
5. In general, when you contact your primary care clinic to get an appointment for care you need right 
away, how often do you get an appointment as soon as you need? [care_need] 
Never 
Sometimes  
Usually  
Always 
Does not apply to me because I never need care right away 
 
6. Thinking about all the care you typically receive from your primary care clinic (including phone, 
clinic, video or home visits)... [phone only]: Read response options aloud for first row, then as 
needed for the rest of the matrix.  
No effort is made 
A little effort is made 
Some effort is made 
A lot of effort is made 
Every effort is made 
 
d. How much effort is made to help you understand your health issues? [effrt_hlth_iss] 
e. How much effort is made to listen to the things that matter most to you about your health 
issues? [effrt_lstn] 
f. How much effort is made to include what matters most to you in choosing what to do next? 
[effrt_nxt_step] 
 
7. Receiving health care often means seeing different people, such as office staff, nurses, doctors, and 
other health professionals. Please think about a health issue that leads you to see different health 
professionals and answer the following questions. [phone only]: Read response options aloud for 
first row, then as needed for the rest of the matrix.  
1, Never 
2, A little 
3, A lot 
4, Always 
 
e. How often do you have to do or explain something because people do not share information 
with each other? [ct_info_share] 
f. How often are you confused because people give you conflicting information or advice? 
[ct_info_conf] 
g. How often do you feel uncomfortable because people do not get along with each other? 
[ct_get_alng] 
h. How often are you unclear whose job it is to deal with a specific question or concern? 
[ct_wrk_flow] 
 
 
8. Have you received help with any of the following from your clinic in the last year? (Please check all 
that apply). [phone only]: Read each response option aloud, marking all responses participant says 
“yes.” [rcvd_help] 
____ Getting medical or mental health services by specialists 
____ Getting medical or mental health services by primary care clinicians 
____Transitions in care (example – from hospital to home) 

Appendix H 
MNCARES Primary Cohort patient survey 
Version 1.2, May 1, 2023 
120 
____ Managing chronic medical problems 
____ Health education or health behavior counseling 
____ Identifying financial needs and resources  
____ Identifying mental health or emotional needs and resources  
____ Identifying spiritual needs and resources 
____ Identifying housing 
____ Identifying transportation 
____ Identifying food needs and resources 
____Identifying other community resources 
____ Other (please explain) ____________________________________ 
[rcvd_help_oth] 
 
The next questions are about care coordination. A care coordinator is a nurse, social worker, or other 
medical professional who helps patients manage their health or healthcare services. They can help 
with things like coordinating doctor's appointments or providing referrals for resources. This person 
might also be called a care manager, care navigator or health coach.  
9. Have you ever had any type of visit (including phone, clinic, video or home) with a care coordinator 
from your clinic?  [care_coord_ever] 
Yes 
No  Skip to Q15 
 
10. [if yes to visit with CC ever] Are your care coordination visits usually in-person or by phone or video 
call? [care_coord_mode] 
Usually or always in person 
Both in person and by phone or video call 
Usually or always by phone or video call 
 
11. [if yes to visit with CC ever] How often did you see or talk to a care coordinator in the first 3 months 
of receiving care coordination? [care_coord_freq] 
More than once a week 
About once a week 
A few times a month 
About once a month 
Less than once a month 
 
12. [if yes to visit with CC ever] What are you and your current care coordinator hoping to achieve 
through your work together? Please check all that apply. [phone only]: Read each response option 
aloud, marking all responses participant says “yes.” [care_coord_hope] 
____ Getting medical or mental health services by specialists 
____ Getting medical or mental health services by primary care clinicians 
____Transitions in care (example – from hospital to home) 
____ Managing chronic medical problems 
____ Health education or health behavior counseling 
____ Identifying financial needs and resources  
____ Identifying mental health or emotional needs and resources  

Appendix H 
MNCARES Primary Cohort patient survey 
Version 1.2, May 1, 2023 
121 
____ Identifying spiritual needs and resources 
____ Identifying housing 
____ Identifying transportation 
____ Identifying food needs and resources 
____Identifying other community resources 
____ Other (please explain) [care_coord_hope_oth] 
____ Don’t know 
13. [if yes to visit with CC ever] On a scale from 0 to 10, where 0 is the worst care coordinator possible, 
and 10 is the best care coordinator possible, what number would you use to rate your current care 
coordinator? [care_coord_rate] 
 
0 Worst care coordinator possible 
 
1 
 
2 
 
3 
 
4 
 
5 
 
6 
 
7 
 
8 
 
9 
 
10 Best care coordinator possible 
14. [if yes to visit with CC ever] Have you received any help or services from a social worker at your clinic 
since starting care coordination? [care_coord_sw] 
 
Yes 
 
No 
 
15. Please think about all of the things that you must do to take care of your health because of an 
ongoing illness or health condition, such as seeking and understanding medical information, taking 
medications, going to medical appointments, monitoring your health, diet, and exercise. 
On a scale of 0 to 10, with 0 being very easy and 10 being very difficult, over the past 4 weeks, how 
easy or difficult has it been for you to do all of the self-care tasks that you need to do?  
[burden_work] 
0 
1 
2 
3 
4 
5 
6 
7 
8 
 
9 
10 
Very 
easy 
 
 
 
 
Neither 
easy 
nor 
difficult 
 
 
 
 
 
Very 
difficult 
 
16. On a scale of 0 to 10, with 0 being not at all and 10 being very much, over the past 4 weeks, how 
much have these self-care tasks interfered with your everyday activities and your overall enjoyment 
of life? [burden_impact] 
0 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 

Appendix H 
MNCARES Primary Cohort patient survey 
Version 1.2, May 1, 2023 
122 
Not at 
all 
 
 
 
 
Somewhat 
 
 
 
 
Very 
much 
 
Finally, we have some questions about you and your living situation. These will help us better 
understand who answered this survey. 
17. Which of the following best describes your current living situation? [lvng_now] 
I have a steady place to live 
I have a place to live today, but I am worried about losing it in the future 
I do not have a steady place to live (temporarily staying with others, in a hotel, in a shelter, living 
outside on the street, on a beach, in a car, abandoned building, bus or train station, or in a park) 
 
 
18. In the last 12 months, how often did you or other adults in your household ever eat less at a meal or 
skip meals because there wasn’t enough money for food? [lvng_meal] 
Often 
Sometimes 
Rarely 
Never 
 
19. In the last 12 months, has lack of reliable transportation kept you from medical appointments, 
meetings, work or from getting things needed for daily living? [transpo] 
Yes 
No 
 
20. Do you need help with day-to-day self-care activities such as bathing, preparing meals, shopping, 
managing finances, etc..? [daily_help] 
1, Yes 
0, No  Skip to question Q22 
 
21. If Q20 =1 [If need help] Do you get the help with day-to-day self-care activities you need? 
[daily_help_need] 
Yes, definitely 
Yes, somewhat 
No 
 
22. When you need advice or support are there family or friends you can turn to? [support_grp] 
Yes 
No  
Don’t know 
 
23. How often do you feel lonely or isolated from those around you? [isolate] 
Never 
Rarely 
Sometimes 
Often  
Always 
 

Appendix H 
MNCARES Primary Cohort patient survey 
Version 1.2, May 1, 2023 
123 
24. In the last 12 months, was there any time when you needed medical care, but did not get it because 
you couldn’t afford it? [affordcare] 
Yes 
No  
 
25. In the last 12 months, was there any time when you needed prescription medicines, but did not get 
it because you couldn’t afford it? [rx_afford] 
Yes 
No  
 
26. Approximately how much did anyone in your family pay ‘out-of-pocket’ (that is, including co-pays, 
coinsurance, and meeting of deductibles) for health care services you received in the last 12 
months? [pkt_cost] 
Zero 
Less than $500 
$500-$1,999 
$2,000-$4,999 
$5,000 or more 
 
27. Do you currently have any kind of health care coverage, including health insurance, prepaid plans 
such as HMOs, or government plans such as Medicare, or Indian Health Service? [cover_info] 
Yes 
No 
 
28. How did each of the following change as a result of working with a care coordinator? Would you say 
it is…  
Much worse  
Somewhat worse  
Not different  
Somewhat better  
Much better 
Not applicable 
a. Ease or difficulty of your self-care tasks [cc_change_ease] 
b. Interference of self-care tasks on your everyday activities and your overall enjoyment of life 
[cc_change_interfere] 
c. Stability of your living situation [cc_change_living] 
d. Your health care coverage [cc_chane_coverage] 
 
29. How has the frequency of each of the following changed as a result of working with a care 
coordinator? Would you say it occurs… [phone only]: Read response options aloud for first row, then 
as needed for the rest of the matrix. 
Much more often 
Somewhat more often 
No change 
Somewhat less often 
Much less often 
Not applicable 

Appendix H 
MNCARES Primary Cohort patient survey 
Version 1.2, May 1, 2023 
124 
a. Frequency that you or other adults in your household eat less or skip meals because there 
wasn't enough money for food [cc_freq_meals] 
b. Frequency that you had lack of reliable transportation [cc_freq_transport] 
c. Frequency that you get the help that you need [cc_freq_help] 
d. Frequency that you did not get needed prescriptions or medical care because you could not 
afford it [cc_freq_med] 
 
We have just a few more questions about you. 
 
30. Do you currently live alone? [lvng_sit_alone] 
Yes 
No 
 
31. If Q30=0 [If do not live alone] How many members of your household, including yourself, are 18 or 
older? [lvng_18up] 
_ _ Number  
 
  
32. If Q30=0 [If do not live alone] How many members of your household are under 18? [lvng_18less] 
 _ _ Number  
 
33. What is your marital status? [marital_stat] 
Now married 
Widowed 
Divorced 
Separated 
Never married 
 
34. Are you a caregiver for a child or other friend or family member? Caregiving may include help with 
personal needs or household chores. It might be managing a person’s finances, arranging for outside 
services, or visiting regularly to see how they are doing. This person need not live with you. 
[caregiver] 
Yes 
No 
 
35. Do you live with a cat, dog or other pet that requires your care? [pet] 
Yes 
No 
 
36. What is the highest grade or year of school you completed? [edu_comp] 
8th grade or less 
Some high school 
High school graduate 
Some college or technical school 
College graduate 
Some graduate school or more 
 
37. Are you currently….? (If more than one applies, select the category which best describes you.) 
Employed for wages  

Appendix H 
MNCARES Primary Cohort patient survey 
Version 1.2, May 1, 2023 
125 
Self-employed  
Out of work for 1 year or more  
Out of work for less than 1 year  
A homemaker  
A student  
Retired 
Unable to work 
 
38. What is your best estimate of the total income of all family members living in your household from 
all sources before taxes in the last calendar year?  
Less than $15,000 
Between $15,000 and $29,999 
Between $30,000 and $49,999 
Between $50,000 and $99,999 
$100,000 or more 
 
39. Is there anything else that you would like to tell us about your responses or about your experiences 
with health and health care or how things may have changed as a result of working with a care 
coordinator? 
_________________________ 
_________________________ 
_________________________ 
 
40. Would you like to receive an email with summary results from this survey and what we learn about 
care coordination when available?  
 
Yes  What is your email address? _______________________ 
 
No 
 
To thank you for your time, we would like to send you a $20 gift card of your choice.  
- 
Select the gift card you would like to receive: 
o Target 
o Walmart 
- 
Please enter the best mailing address to send your gift card 
• 
Address 1: 
• 
Address 2: 
• 
City: 
• 
State: 
• 
Zipcode: 
 
Thank you for taking the time to complete this survey. Your responses are very important. 

Appendix I 
MNCARES Historical Cohort Phase 1 patient interviews 
Version 1.2, May 1, 2023 
126 
17.9 Appendix I: Historical cohort Phase 1 patient interviews 
Phone script 
 
Hello, is [first_name] [last_name] available? 
1, Yes, continue to A 
0, No   Is there a better time to call back? (Or call back in 10 minutes if they say “I don’t 
know”.) 
 
If you have established that you are speaking to the correct participant, say:  
 
A. Hello, my name is [interviewer first name], and I’m calling from HealthPartners Institute about the 
Care Coordination interview we have scheduled for today. Is now still a good time to do the interview? 
1, Yes, continue to B 
0, No   when is a better time to call back? [Reschedule interview and end call – or schedule a 
time to call back to reschedule if time does not permit rescheduling now.] 
 
B. Thank you for agreeing to participate in this interview. Your thoughts and feedback will be very 
helpful to other patients in the future. The interview will last about 30 to 45 minutes and will be 
recorded and transcribed by an outside company. Your participation in the interview and your responses 
to interview questions will not be shared with your clinic, your insurance, or your health care providers. 
All information, including your name and responses, will be kept confidential. You will receive a $35 gift 
card for your participation in the interview today.  
 
As a reminder, you were chosen for the interview because of the care coordination services you have 
received from your clinic. A care coordinator is a nurse or a social worker who helps patients manage 
their health or healthcare services. They can help with things like coordinating doctor’s appointments or 
providing referrals for resources. This person might also be called a care manager or care navigator. 
 
Also wanted to mention I will be taking notes to make sure I’m catching all the detail you provide in case 
you hear my keyboard in the background. 
 
(Note: Keep track of what language patients use to describe care, their clinic vs their doctor’s office.)  
 
Clinics throughout Minnesota are trying to understand how the COVID-19 pandemic has impacted 
people’s health problems and the health care they receive. Even if they had not been sick with COVID-
19, a lot of things may have changed for many people in their life, health, and experiences with health 
care. The information you give us today will be used to provide clinics with the information they need to 
provide better care.  
 
C. Do you have any questions before we begin? 
1, Yes, record questions and answer using FAQ 
 
0, No   Begin interview 
 
I will begin the recording now. 
 
 

Appendix I 
MNCARES Historical Cohort Phase 1 patient interviews 
Version 1.2, May 1, 2023 
127 
Question guide 
Introductory Question/s about Impact of COVID-19 (10-15 min) 
Many of the questions I ask you today will have to do with receiving care for your health problems. Before 
we get started, could you share from your perspective, what are your two most important health 
problems? 
Note: The purpose of this question is to set the stage for the interview and to see how easily participants 
can answer this item, which will be used in the survey. In depth probing is not needed, unless the participant 
does not understand the question. Make note of what language is used by participants to describe “health 
problems” or health conditions. 
1. Tell me a little bit about how the COVID-19 pandemic has affected your life in general.  
  Note: Start with COVID-19 then use whatever terminology the participant uses. 
Probes 
Note: Probe for additional impacts, generally. 
1. What other types of effects has COVID-19 had on your life?  
Note: If not able to come up with additional impacts, use the topics below (depending on 
prior responses). 
Topic: Impact on Social Connections 
- 
The pandemic has impacted many people’s relationships with friends and family. How has the 
pandemic affected your relationships? 
- 
How did your day-to-day interactions or outings change with the pandemic? 
Note: Be careful not to suggest impacts and stick closely to language provided here, without 
improvising (eg, don’t ask if they feel ‘lonely’, let them use their own words), but probe further 
using the language the respondent offers.  
Topic: Impact on Health Status 
- 
How has the pandemic impacted your overall health? 
- 
Has the pandemic affected the health conditions you mentioned above? 
Note: This section and specifically this second probe are highest priority for this section. 
Topic: Impact on Finances/Employment  
2. Has the COVID-19 pandemic had any impact on your work or other responsibilities? 
 
Note, only ask the below question for those who have mentioned above they are working. 
3. How has the COVID-19 pandemic affected your ability get to work or do your work?  
4. Has the COVID-19 pandemic affected your financial situation in any way? 
5. Has the COVID-19 pandemic affected any government support you might receive?  
Topic: Impact on Mental Health 
- 
Do you think the pandemic has affected your mental health in any way?  
- 
What stresses have you felt related to COVID-19?  
- 
Tell me about the stresses you have experienced and how this affected your health or 
mental health. 
- 
Change and stress can affect mental health for many people. What types of changes have you 
experienced related to the COVID-19 pandemic? 
- 
Has it affected how you interact with others? If yes, How so? 
- 
Has it changed how you interact with family members or friends in any way? If yes, How so? 
Note: Be very careful here to show no judgement on responses or current behaviors; if the 
participant has an extreme response, be neutral, de-escalate, re-frame the question. 
Note: If talking about impact on healthcare or care coordination, unprompted, go on to the next 
section and return after completed. 
2. Is there anything specific that made the last year harder for you than it was for others? 

Appendix I 
MNCARES Historical Cohort Phase 1 patient interviews 
Version 1.2, May 1, 2023 
128 
Impact of COVID-19 on Healthcare (15-20 min) 
3. What difficulties have you had getting health care during the pandemic? 
Probes 
- 
Are you able to access providers and services when you need to? 
- 
How often do you get care? What are the main reasons you go to the clinic or doctor’s office? 
How did this change with the pandemic? 
Note: If not offered, probe to see if care was delivered virtually via video visits or phone and how 
that experience was compared to prior care. Probe about technology needs, internet connection, 
etc. 
- 
Have you done any video visits? How was your experience with video visits? How was 
this compared to before the pandemic? 
- 
Is there anything about the way you received healthcare during the pandemic was particularly 
troublesome? Helpful? 
Note: Follow up with general probes to learn more details. 
- 
Does anyone come into your home to help you? If yes, How has this been affected by the 
pandemic? 
- 
Were you able to get in to the clinic when needed or reach providers you needed to see?  
- 
Did talking with your care coordinator change in any way because of the COVID-19 pandemic?  
4. What types of things do you wish your clinic had done to help you during this time? Is there other 
support you wish you had during this time? 
Impact on COVID-19 Care Coordination (5 min) 
5. I know you have met with a care coordinator, how did talking with this person help you manage your 
health and health care during this time of COVID? Note: If they use the first name or other alias of their 
care coordinator, repeat this throughout. 
Probes 
- 
Did anything you and your care coordinator did before COVID help you manage changes in 
the last year? 
Note: Add branching so this question only appears if they’ve had CC during COVID-19 (collected in the screener) 
6. How has COVID-19 affected the coordination of care you receive/d from your clinic, specifically your 
experience with your care coordinator or social worker?  
Probes 
- 
What’s happening right now with your care coordinator? 
7. I want you to think back before COVID-19 started in March of 2020. Tell me a little bit about the care 
and care coordination you were receiving then. 
Probes 
- 
Did your care coordinator help you deal with any of the challenges or stresses of COVID-19 
that you mention earlier? How so? 
- 
Did the care coordination services you received help you deal with these impacts or stresses? 
Note: Add branching so this question only appears if they’ve had CC during COVID-19 (collected in the screener) 
8. So now that we’ve talked about what’s happening with your care coordinator currently and what 
happened before the COVID-19 pandemic, could you tell me a little bit about how has this changed 
for you? 
9. Is there anyone else in your life helping you manage your health or healthcare services right now? 
Probes 
If yes, 
- 
Does anybody else live in your household?  
- 
Tell me a bit about that person and how they are helping you. 
- 
What things are they helping you with? 
Cool-down Question/s  

Appendix I 
MNCARES Historical Cohort Phase 1 patient interviews 
Version 1.2, May 1, 2023 
129 
10. When you talk about care coordination with your family and friends, how do you describe it to them? 
11. Is there anything else you’d like me to know about how COVID-19 has impacted your life, your health 
care or care coordination in general? 
 
D. I have stopped the recording. Thank you so much for completing this interview.  
 
Could you please provide your mailing address so that we can send you your gift card? 
Address1 Address2 
City, State Zip 
 
Can we contact you in the future for things like testing out new surveys or media opportunities related 
to this study (for example, news articles written about the study)?  
1, Yes, record response  
 
0, No, record response 
 
Would you like us to send you a summary of what we learned from these interviews or the findings in 
our larger research study?  
1, Yes, record response, continue 
 
0, No, record response 
 
If yes to one of the above questions: What is your preferred method for contact in the future? 
 
1, Email 
2, Phone 
3, Postal mail 
 
We will send you your gift card soon. We appreciate you taking the time to participate in this important 
project/study.  

Appendix J 
MNCARES Primary Cohort Phase 1 patient interviews 
Version 1.2, May 1, 2023 
130 
17.10 Appendix J: Primary cohort Phase 1 patient interviews 
Phone Script 
 
Use the script as a guide, be engaging in your speech so the participant feels a sense of rapport and 
comfort from the very beginning. 
 
Hello, is [first_name] [last_name] available? 
1, Yes, continue to A 
0, No   Is there a better time to call back? (Or call back in 10 minutes if they say “I don’t 
know”.) 
 
If you have established that you are speaking to the correct participant, say:  
 
A. Hello, my name is [interviewer first name], and I’m calling from HealthPartners Institute about the 
Care Coordination interview we have scheduled for today. Is now still a good time to do the interview? 
1, Yes, continue to B 
0, No   when is a better time to call back? [Reschedule interview and end call – or schedule a 
time to call back to reschedule if time does not permit rescheduling now.] 
 
B. Thank you for agreeing to participate in this interview. Your thoughts and feedback will be very 
helpful to other patients in the future. Your thoughts will be used to design a statewide survey about 
care coordination services to help us learn how to improve care. The interview will last about 20 to 30 
minutes and will be recorded and transcribed by an outside company. This is not a satisfaction survey 
and will not be used to evaluate your care coordinator, doctors, or clinic. Your participation in the 
interview and your responses to interview questions will not be shared with your clinic, your insurance, 
or your health care providers. All information, including your name and responses, will be kept 
confidential. You will receive a $20 gift card for your participation in the interview today.  
 
As a reminder, you were chosen for the interview because of the care coordination services you have 
received from your clinic. A care coordinator is a nurse or a social worker who helps patients manage 
their health or healthcare services. They can help with things like coordinating doctor’s appointments or 
providing referrals for resources. This person might also be called a care manager or care navigator. 
 
I also wanted to mention I will be taking notes to make sure I’m catching all the detail you provide in 
case you hear my keyboard in the background. 
 
Before we get started, I wanted to let you know that I am not an expert in care coordination. My 
expertise is in asking people questions. You are the expert in your experience and I'm here to learn from 
you. 
 
(Note: Keep track of what language patients use to describe care, their clinic vs their doctor’s office.)  
 
C. Do you have any questions before we begin? 
1, Yes, record questions and answer using FAQ 
 
0, No   Begin interview 
 
I will begin the recording now. You will hear a voice telling you the recording is starting.  

Appendix J 
MNCARES Primary Cohort Phase 1 patient interviews 
Version 1.2, May 1, 2023 
131 
Question guide 
 
C Experience (7 min) 
Topic: General experience 
1. What has your experience with care coordination been like? 
Probe/s: 
- 
Tell me about your last visit with your care coordinator? What did you talk about? 
- 
Have you worked with more than one care coordinator? 
- 
Has it been helpful? In what ways? What has been most helpful? 
2. What were the main reasons you started or were referred to care coordination at your clinic?  
Probe/s:  
- 
What health needs/issues are you trying to improve with care coordination? 
3. What services and benefits have you received from having a care coordinator? 
Probe/s:  
- 
Are there any other services or support you wish you had received from your Care 
Coordinator? 
- 
Are there any other issues you are having with your health care that you haven’t talked 
about with your CC? 
- 
Have there been any barriers to using CC services? 
- 
What has made using CC services easier? 
Communication (5 min) 
Topic: CC Communication 
4. What is your relationship like with your care coordinator? 
Probes/s 
- 
How do you communicate with your CC? How well does that work? 
- 
How often do you have contact with the care coordinator? Usually in person? What other 
ways -- phone, portal, video or other ways? 
- 
Does your CC ever talk to your family or caregivers? 
- 
Is there anything you CC does to show they care about you? Is there anything you wish 
they would do? 
- 
Does it feel like your CC has time for you? 
Topic: Care Team Communication 
5. Does your doctor seem to know what you talk with your care coordinator about? Do they know the 
important pieces relevant to your care? 
Probes/s 
- 
Does your doctor seem informed about the work you are doing with your care 
coordinator? 
- 
Do you care coordinator and doctor/s communicate well in the coordination of your care? 
What has given you this feeling/impression? Can you give an example of how they 
communicate?   
- 
Do they seem to be working together for your care? 
Patient Needs Met (5 min) 
6. How is your health now compared to before you started care coordination? Do you think this 
change is because of care coordination? 
Probe/s:  
- 
Is it easier or harder to manage your health now? Do you think this change is because of 
care coordination? 
- 
To what extent has your care coordinator helped you to deal with your health problems?  

Appendix J 
MNCARES Primary Cohort Phase 1 patient interviews 
Version 1.2, May 1, 2023 
132 
- 
Have they helped you with other needs? How so?  
Survey Response (2 min) 
As a reminder, the information you’ve shared today will be used to design a statewide survey about care 
coordination services to help us learn how to improve care. We’re trying to problem solve how to best 
reach patients and would love your help brainstorming. 
7. What do you think would make patients more likely to open a letter? What might make them more 
likely to fill out and return a survey? 
Probes/s 
- 
In the past have you gotten surveys in the mail? Do you usually open mail where you don’t 
know the sender or think it’s a survey? What led you to/not to complete it?  
- 
If you were to receive a mailed invitation to complete a survey about care coordination, 
what might make you more likely to open the envelope or to agree to complete the 
survey? If you saw something with a MNCARES logo would you open it? Would the return 
address or anything about the envelope make a difference? 
- 
Do you know what a QR code is? Do you know how to use them? Do you use them? What 
makes you more likely.   
- 
Would it make a difference who signed the invitation? What the purpose was? 
Cool-down Question (1 min) 
8. Is there anything else you’d like me to know about your health care management or care 
coordination? 
 
I have stopped the recording. Thank you so much for completing this interview. Your answers will help 
improve care for other patients.  
Could you please provide your mailing address so that we can send you your <gift card/check/cash>? 
Address1 Address2 
City, State Zip 
 
Can we contact you in the future for things like testing out new surveys or media opportunities related 
to this study (for example, news articles written about the study)?  
1, Yes  
 
0, No 
 
Would you like us to send you a summary of what we learned from these interviews or the findings in 
our larger research study?  
1, Yes 
 
0, No 
 
If yes to one of the above questions: What is your preferred method for contact in the future? 
1, Email (Check to make sure email address was collected during screening.) 
2, Phone 
3, Postal mail 
 
We will send you your <gift card/check/cash> soon. We appreciate you taking the time to participate in 
this important interview.  
 

Appendix K 
MNCARES Care coordinator Phase 1 interviews 
Version 1.2, May 1, 2023 
133 
17.11 Appendix K: Care coordinator Phase 1 interviews 
Phone script 
Hello, is [first_name] [last_name] available? 
1, Yes, continue to A 
0, No   Is there a better time to call back? (Or call back in 10 minutes if they say, “I don’t 
know”.) 
 
If you have established that you are speaking to the correct participant, say:  
 
A. Hello, my name is [interviewer first name], and I’m calling from HealthPartners Institute about the 
Care Coordination interview we have scheduled for today. Is now still a good time to do the interview? 
1, Yes, continue to B 
0, No   when is a better time to call back? [Reschedule interview and end call – or schedule a 
time to call back to reschedule if time does not permit rescheduling now.] 
 
B. Thank you for agreeing to participate in this interview. As you know, we are working with your clinic 
and others to learn about how care coordination services are provided at your clinic and what is 
important for effective care coordination. We will be using this information to help us design a survey 
for care coordinators in many other clinics. The interview will take 30-40 minutes. Interviews will be 
recorded and transcribed by an outside company. Your participation in this interview is completely 
voluntary. Any information you provide will not be shared with your clinic, care system or the Minnesota 
Department of Health in a way that you could be identified. All information, including your name and 
organization will be kept confidential. 
 
Also wanted to mention I will be taking notes to make sure I’m catching all the detail you provide in case 
you hear my keyboard in the background. 
 
C. Do you have any questions before we begin? 
1, Yes, record questions and answer using FAQ 
 
0, No   Begin interview 
 
I will begin the recording now. 
 
Question guide 
 
General, Open (2 min) 
I’m going to start with a few questions to help us understand how coordinator services are provided at 
your clinic. 
1. First, tell me a bit about your overall role as a care coordinator, what does it involve? 
Probe/s:  
- 
What else are care coordinators at your clinic responsible for? 
CC Model (7 min) 
Note, the primary purpose of this section is to understand if the care coordinator is able to describe the 
care coordination model at their clinic, according to MNCARES definitions and if they are able to talk 
about the model before and after COVID.  
2. Who is a part of the care coordination team at your clinic? 

Appendix K 
MNCARES Care coordinator Phase 1 interviews 
Version 1.2, May 1, 2023 
134 
Probe/s:  
- 
[If no social worker is mentioned] Is there a social worker at your clinic?  
- 
[If a social worker is mentioned or yes to the probe above] How is the social worker/s on 
your team involved in care coordination?  
Additional Probe/s if needed, time available: 
• 
What is their role?  
• 
Do they work directly with care coordination patients?  
• 
Do they work directly with the clinicians or care teams related to specific patients?  
3. If a patient needs social services or resources, how is that facilitated at your clinic?  
Probe/s (if time):  
[If not mentioned above under resources]  
- 
What if a patient has needs for transportation? Financial help? Housing? Psychological or 
emotional issues? Behavioral health issues?  
[If both nurses and social workers provide care coordination services at this clinic]  
- 
How do nurses and social workers differ in providing resources or services at your clinic? 
As I mentioned we will be creating a survey from our interviews, this next question relates to the 
timeframe we are interested in, in addition to current care coordination services: 
4. Would you be able to describe the care coordination model in your clinic as far back as 2018?  
Probes:  
- 
If you weren’t in this clinic at that time, is there someone at your clinic you could ask to help 
you answers questions about this on a survey?  
- 
Do you know if there was a social worker on the care team in 2018?  
• 
If so, has their role changed?  
5. Thinking about the coming year, do you expect any changes to how care coordination is delivered 
at your clinic?  
- 
If so, what type of changes do you anticipate?  
CC Components (5 min) 
6. Tell me a bit about your interactions with patients as a care coordinator, how are they 
structured?  
Probes (if time): 
- 
Does this vary by patient?  
- 
What are your relationships like with your care coordination patients? 
- 
How do you communicate with your care coordination patients? 
- 
What are your thoughts about in-person versus virtual communications with care 
coordination patients? 
- 
How do you interact with clinicians about care coordination services their patients are 
receiving?  
- 
How often do you communicate with clinicians? 
7. How are patients identified and enrolled in care coordination at your clinic? 
Factors for Effective CC (5 min) 
8. What do you think are the most important components for effective care coordination in your 
clinic? 
Probes (if time): 
- 
What backgrounds and experience are important on a care coordination team? 
- 
What do you think is the optimal panel size for a care coordinator?  
- 
What kinds of organizational resources are needed for effective care coordination?  

Appendix K 
MNCARES Care coordinator Phase 1 interviews 
Version 1.2, May 1, 2023 
135 
9. If you could add another role to your care coordination team, what would it be? 
Barriers and Facilitators of Effective CC (5 min) 
10. What are the main barriers to effective care coordination in your clinic?  
Probe/s: 
- 
Are there things that get in the way in providing effective care coordination? 
- 
Are their things in your clinic that are particularly helpful to you in providing effective care 
coordination? 
Pts best served by CC (5 min) 
11. What kinds of patients do you think benefit the most from care coordination? 
Probe/s: 
- 
Are there particular conditions, or situations that benefit the most? 
General, close (1 min) 
12. Is there anything else that you think we should know about that is important for providing 
efficient and effective care coordination services in primary care? 
 
I have stopped the recording. Thank you so much for completing this interview. 
 
Can we contact you in the future for things like testing out new surveys or media opportunities related 
to this study (for example, news articles written about the study)? 
1, Yes 
0, No 
 
Would you like us to send you a summary of what we learned from these interviews or the findings in 
our larger research study? 
1, Yes 
0, No 
 
What is your preferred method for contact in the future? 
1, Email (Check to make sure email address was collected during screening.) 
2, Phone 
3, Postal mail 
 
We appreciate you taking the time to participate in this important interview. 
 

Appendix L 
MNCARES Clinican and leader Phase 1 interviews 
Version 1.2, May 1, 2023 
136 
17.12 Appendix L: Clinician and leader Phase 1 interviews 
Phone script 
 
A. Hello, is [first_name] [last_name] available? 
1, Yes, continue to A 
0, No   Is there a better time to call back? (If “don’t know” then, say: I’ll call back in 10 
minutes.) 
 
If you have established that you are speaking to the correct participant, say:  
 
B. My name is [interviewer first name], and I’m calling from HealthPartners Institute about the 
interview we have scheduled for today. Is now still a good time to do the interview? 
1, Yes, continue to C 
0, No   When is a better time to call back? [Reschedule interview and end call – or schedule a 
time to call back to reschedule if time does not permit rescheduling now.] 
 
C. Thank you for agreeing to participate in this interview. We are working with your clinic and others to 
learn about how care coordination services are provided at your clinic and what factors you think 
are important for successful care coordination.  
 
[If asked for a definition of care coordination: By care coordination we mean a program where a 
nurse or social worker helps patients manage their health or healthcare services, coordinating 
appointments or providing referrals for resources. A care coordinator might also be called a care 
manager or care navigator at your clinic.]  
 
We will use the information you share to design a statewide survey for care coordinators. The 
interview will take about 20 minutes. Interviews will be recorded and sent to an outside company 
for transcription. You can choose if you want to participate in these interviews or not and any 
information you provide will not be shared with anyone in your clinic or care system. Any 
information we report or publish will not be linked to you or your clinic. All information, including 
your name and answers, will be kept confidential. 
 
D. Are you OK with me recording our conversation? 
1, Yes  Continue to E 
0, No   Unfortunately we will not be able to continue the interview without recording. Thank 
you for your interest in participating. End interview and document 
 
Also wanted to mention I will be taking notes to make sure I’m catching all the detail you provide in 
case you hear my keyboard in the background. 
 
E. Do you have any questions before we begin? 
1, Yes, record questions and answer using MNCARES FAQ, direct to study email, begin interview 
 
0, No   Begin interview 
 
I will begin the recording now.  
 

Appendix L 
MNCARES Clinican and leader Phase 1 interviews 
Version 1.2, May 1, 2023 
137 
Question guide 
General, Open (3 min) 
I’m going to start with a few questions to help us understand a bit about care coordinator services at 
your clinic. 
1. Can you tell me a bit about your experience with care coordination? 
Probe/s:  
- 
For clinicians or NPs: How long have you utilized care coordination? 
- 
For clinicians or NPs: How often do you use care coordination? 
- 
For clinicians or NPs: How many of your patients are in care coordination currently? 
- 
For clinic managers/leads: About how many of the patients in your clinic are in ongoing care 
coordination? 
- 
Has your experience been primarily in this clinic or also in other settings? 
CC Model, Components (5 min) 
2. Now I’d like to hear more about the care coordination processes at your clinic. What are care 
coordination services like in your clinic now? 
Probe/s:  
- 
To what extent does your CC/team address the social needs of patients? 
- 
In your clinic, how do patients start receiving care coordination services? 
- 
What types of situations prompt a referral to care coordination? 
- 
Under what circumstances do patients generally stop receiving care coordination services? 
- 
How long do most patients stay in care coordination in your clinic? 
3. In your clinic, how do care coordinators communicate with clinicians about their patients? 
Probe/s: 
- 
How well does this approach work for clinicians? 
- 
How well does this approach work for patients? 
- 
What could be improved? 
Factors for Effective CC (5 min) 
4. How helpful do you think care coordination is for patients at your clinic? 
Probe/s: 
- 
What do you think are the main benefits of care coordination for patients? 
- 
How satisfied have patients been with the way care coordination is being done at your clinic? 
- 
What do you think are the main reasons patients are either satisfied or dissatisfied? 
5. In general, what are the most important features of successful care coordination?  
Probe/s: 
- 
What does successful care coordination look like at your clinic? 
- 
What kinds of coordination services are most important? 
- 
What kinds of patients benefit the most from coordination? 
- 
What are the most important characteristics for a care coordinator to have? Do you think it 
is important for care coordinators to have a social work degree? 
- 
How often should coordination patients connect with their care coordinator? 
Barriers for Effective CC (5 min) 
6. What do you think are the main barriers to successful care coordination? 
Probe/s: 
- 
If you had the power to change the current approach to care coordination, what changes 
would you make? 
General, Close (2 min) 
7. Is there anything else you think we should know about how care coordination works? 

Appendix L 
MNCARES Clinican and leader Phase 1 interviews 
Version 1.2, May 1, 2023 
138 
 
I have stopped the recording. Thank you very much for your time and your ideas. They will be very 
helpful as we learn how to make coordination more successful. 
Can we contact you in the future for things like testing out new surveys or media opportunities related 
to this study (for example, news articles written about the study)? 
1, Yes 
0, No 
 
Would you like us to send you a summary of what we learned from these interviews or the findings in 
our larger research study? 
1, Yes 
0, No 
 
What is your preferred method for contact in the future? 
1, Email (Check to make sure email address was collected during screening.) 
2, Phone 
3, Postal mail 
 
Thank you so much for completing this interview. 
 
 
 
 
 

Appendix M 
MNCARES Study Organizational Chart 
Version 1.2, May 1, 2023 
139 
17.13 Appendix M: Study Team Organizational Chart