Improving Care Coordination for Children With Disabilities Through an Accountable Care Organization
Conditions: Experiences With Health Care Coordination
Sponsor: University of North Carolina, Chapel Hill

 
Improving Care Coordination for Children with Disabilities through an 
Accountable Care Organization 
 
 
 
NCT number NCT02304380 
Document Date February 17, 2015 
 

rev. 02/17/2015 
 
Improving Care Coordination for 
Children with Disabilities through an 
Accountable Care Organization 
 
 
 
STUDY PROTOCOL 
Version 2 
 
 
Principal Investigator 
Paula Song, PhD 
 
Co-Investigators 
UNC Chapel Hill 
Marisa Domino, PhD 
 
Ohio State University 
Brian Hilligoss, PhD 
Sandra Tanenbaum, PhD 
Wendy Xu, PhD 
 
Nationwide Children’s Hospital 
Deena Chisolm, PhD 
 
Seattle Children’s Hospital 
Rita Mangione-Smith, MD 
 
Project Manager 
Renée Ferrari, PhD 
 
 
Prepared For 
Penny Mohr, MA, IHS 
Senior Program Officer 
Patient-Centered Outcomes Research Institute (PCORI) 
1828 L Street, NW 9th Floor, Washington, DC 20036 

rev. 02/17/2015 
 
List of Acronyms 
 
ABD 
Aged, Blind, Disabled 
ACO 
Accountable Care Organization 
COE4CCN 
Center of Excellence on Quality of Care 
Measures for Children with Complex Needs  
CSRL 
Carolina Survey Research Lab 
FECC 
Family Experiences with Coordination of Care 
NCH 
Nationwide Children's Hospital 
OSU 
Ohio State University 
PFK 
Partners for Kids 
SSI 
Supplemental Security Income 
UNC 
University of North Carolina at Chapel Hill 
 
 
 

rev. 02/17/2015 
 
Contents 
LIST OF ACRONYMS ............................................................................................................................. 2 
STUDY ABSTRACT ................................................................................................................................ 1 
INTRODUCTION ..................................................................................................................................... 2 
BACKGROUND ....................................................................................................................................... 2 
ACCOUNTABLE CARE ORGANIZATION MODEL OF CARE .............................................................................. 3 
OHIO STATE MEDICAID POLICY CHANGE .................................................................................................. 3 
STUDY POPULATION .............................................................................................................................. 4 
STUDY JUSTIFICATION ............................................................................................................................ 4 
PROJECTED USE OF STUDY RESULTS ...................................................................................................... 4 
STUDY DESIGN...................................................................................................................................... 5 
GENERAL APPROACH ............................................................................................................................. 5 
RESEARCH QUESTIONS .......................................................................................................................... 5 
HYPOTHESIS ......................................................................................................................................... 5 
STUDY AIMS .......................................................................................................................................... 6 
AIM 1 
6 
AIM 2 
6 
AIM 3 
6 
METHODS AND DATA ANALYSIS......................................................................................................... 6 
APPROACH TO ANALYSIS: MIXED METHODS.............................................................................................. 6 
AIM 1 
7 
AIM 2 
9 
Caregiver Survey ......................................................................................................................... 10 
Medical Record Abstraction ...................................................................................................... 10 
AIM 3 
11 
DATA SAFETY, MONITORING, AND HANDLING ................................................................................ 13 
AIM 1 
13 
AIM 2 
14 
AIM 3 
14 
PROCEDURES ..................................................................................................................................... 16 
AIM 1 
16 
AIM 2 
20 
AIM 3 
22 
DISSEMINATION PLAN........................................................................................................................ 23 
COLLABORATING ORGANIZATIONS, PERSONNEL, PROJECT ROLES .......................................... 24 
UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL ................................................................................. 24 
OHIO STATE UNIVERSITY ...................................................................................................................... 24 
NCH/PFK .......................................................................................................................................... 24 
PATIENT ADVISORY PANEL – VOICES FOR OHIO’S CHILDREN .................................................................... 25 
SEATTLE CHILDREN’S RESEARCH INSTITUTE........................................................................................... 25 
CAROLINA SURVEY RESEARCH LAB ....................................................................................................... 26 
REFERENCES ...................................................................................................................................... 27 
ATTACHMENTS ................................................................................................................................... 31 

1 
 
Study Abstract 
Children with disabilities have complex healthcare needs requiring multiple providers in 
multiple locations. The lack of coordinated care for this vulnerable population leads to poorer 
outcomes, higher costs, and increased stress and time demands for patients and their 
caregivers. Traditionally, under arrangements known as fee-for-service, there have been no 
financial incentives for providers to coordinate care; however, the Affordable Care Act is 
changing that. Accountable care organizations (ACOs) are groups of healthcare providers that 
organize in new ways to take responsibility for the care of a defined population. ACOs share in 
any savings associated with improved quality and efficiency of the care they provide. Although 
most ACOs currently do not cover children with disabilities, many are considering adding these 
to the populations they serve. Yet we know little about effectiveness of the care coordination 
strategies they employ on children with disabilities.  
The goal of this research is to assess care coordination for and patient-centered outcomes of 
children with disabilities under an ACO as compared with traditional fee-for-service plans. We 
will use a recent policy change in Ohio that mandates children with disabilities move from 
traditional fee-for-service Medicaid plans into managed care arrangements such as ACOs. This 
mandate resulted in 8,000 disabled children automatically becoming part of the nation’s largest 
pediatric ACO.  
We will use multiple methods, including focus groups, interviews, a survey, medical record 
data, and Medicare claims, to compare patient experiences and care under the ACO with 
experiences and care under the previous fee-for-service model. What impact will this research 
have? Our research will inform ACOs about the relative benefits and challenges of coordinating 
care and improving the health outcomes of children with disabilities and will help those 
organizations determine whether or not they can adequately serve the needs of this 
population. In addition, the findings will provide patients and caregivers with valuable 
information that can help them make decisions when faced with an increasingly common 
scenario, for example: “The parents of a child with cerebral palsy receive a letter from their 
state Medicaid program that children are being enrolled in an ‘accountable care organization.’ 
How certain can they be that their child’s care will be improved? What are the problems that 
might occur?”  
We will engage patients, their caregivers, and health system stakeholders throughout the 
research process. Patient advocates have been involved in the design of our study. A patient 
advisory panel comprised of caregivers and advocates of disabled children will guide our project 
by providing advice at quarterly meetings. In addition, we will collect data from more than 
2,800 patient voices through direct study participation. 
 
 

2 
 
Introduction 
Background 
Disability is by definition a significant burden on health and functional status. Childhood 
disability places an additional, developmental burden on children, i.e., it makes it more difficult 
for them to do the important developmental work of childhood. For this reason, children’s 
disabilities are sometimes referred to as developmental disabilities. Development disabilities 
span a wide range of health conditions but commonly include physical disabilities such as 
blindness, cerebral palsy, and spinal cord/ traumatic brain injuries and mental disabilities such 
as autism, mental health disorders and intellectual disabilities.[1,2] Approximately 14% of 
children 18 years and younger in the U.S. have a developmental disability, and the prevalence 
of developmental disabilities has increased by over 17% in the past decade.[3,4]   
 
Disabled children are among the most vulnerable populations because having a disability 
creates a higher risk for poor physical, psychological and social health.[5] In addition to their 
primary disabling condition, children with disabilities are increasingly more likely to have 
multiple chronic conditions and comorbidities such as cardiovascular disease or mental 
illness.[6] For example, children with cerebral palsy commonly have intellectual disabilities, 
vision impairment, epilepsy or other secondary musculoskeletal conditions.[4,7] Consistent 
with the severity of the health condition, children with disabilities utilize health care services 
more frequently and intensively compared to children without disabilities [8]. They also face 
significant barriers to accessing health care, greater out-of-pocket health care costs and poorer 
health outcomes compared to children without disabilities.[9] Childhood disability is estimated 
to result in 66 million restricted activity days per year, 24 million days missed of school, 26 
million physician contacts and 5 million hospital days annually.[10] Furthermore, the prevalence 
of developmental disability is higher among African Americans and children in low-income and 
single family households.[3,10] 
 
Children with disabilities require a wide variety of care and services from health providers, 
social service agencies and their families or caregivers. Families and caregivers also serve as the 
main coordinator of care for children with disabilities, and the burden of care and coordination 
has implications for family welfare. Having a child with disabilities can have negative economic, 
health, and social consequences for families and caregivers. For example, the estimates of 
family costs of caring for a child with disability range from $108 to $8,742 per year and 
caregivers spend 4 to 84 hours per week coordinating care for their children.[11] Even low-
income families with disabled children spend over $500 annually.[12] Parental employment is 
also impacted by a child’s disability. Only 1/3 of households with children with disabilities have 
both parents in the labor force compared to over 50% of all households without disabled 
children.[13] Maternal employment is particularly impacted by a child’s disability.[14] Mothers 
of disabled children are less likely to work outside of the home. Those who do work report 
more hours of missed work due a child’s disability, and mothers that quit their job due a child’s 
disability are more likely to report a lower quality of life. [14–17] Having a child with a disability 
also indirectly impacts maternal employment through effects on maternal health.[18] Mothers 
of children with disabilities are more likely to report their health as poor or declining compared 

3 
 
to fathers, and are more likely to report greater levels of stress and poorer mental health. [19–
22] 
 
Accountable Care Organization Model of Care 
Emerging from the Affordable Care Act, Medicaid Accountable Care Organizations (ACOs) are 
being viewed by several states as a new model of health care delivery that offers significant 
opportunities to improve the quality and efficiency of care.  ACOs are groups of providers that 
take responsibility for the care of a defined population and share in any savings associated with 
improved quality and efficiency of the care they provide.[23] One of the key assumptions of the 
ACO model is that the alignment of financial and quality incentives will result in improved 
patient outcomes and efficiency.[24] Among other things, ACOs are expected to integrate high-
risk populations, such as disabled children, that were previously excluded by Medicaid managed 
care arrangements to achieve the goals of higher quality care and improved efficiency through 
care coordination.[25–27] Unlike managed care arrangements, ACOs assume financial risk if 
they do not meet both cost and quality targets. Thus, the ACO model is designed to improve 
coordination of health services and, in turn, improve efficiency and outcomes through the 
presumption of financial risk. Failure to achieve these goals will not only result in financial 
losses to the ACO, but also increase the likelihood that ACOs will exit the market or exclude 
populations that they cannot effectively manage – the very populations who could benefit most 
from coordinated care. In contrast, traditional fee-for-service arrangements do not provide 
financial incentives for providers to coordinate care, so while disabled children may receive 
some care coordination services, these tend to be ad hoc and time-limited. In most cases, a 
parent assumes the responsibility of coordinating medical services for the disabled child. As a 
children’s hospital CEO and mother of a disabled child recently explained, “There is no other 
single person other than my husband who is really paying attention to how care is coordination 
for a kid like Katie.”[28] While the ACO model is touted as a potential solution to the critical 
need for coordinated care, the existing literature speaks more to the promise of ACOs than to 
any actual ACO results.[26,27] At the same time, the ACO model is being actively promoted by 
policy-makers in Ohio and elsewhere as a way to improve quality and lower costs in the state 
Medicaid programs [29–31]. This study will therefore fill a significant gap in the literature by 
comparing the performance of an operating children’s ACO with the past performance of the 
Medicaid fee-for-service system for disabled children.   
 
Ohio State Medicaid Policy Change 
Prior to July 2013, children with disabilities were eligible for traditional, fee-for-service 
Medicaid. In July 2013, all ABD children in Ohio were required to enroll in managed care.  
The policy change effectively moved approximately 8,000 Medicaid-eligible ABD children into 
the Partners for Kids (PFK) ACO Partners for Kids (PFK) is owned by Nationwide Children’s 
Hospital, and assumes responsibility for approximately 300,000 Medicaid children in central 
and southeastern Ohio (34 of 88 counties). 
 

4 
 
Study Population 
Defining the Study Population:  
The Social Security Administration (SSA) considers a child under 
age 18 disabled if the child 1) is not working at a job considered 
to be substantial work; 2) has a physical or mental condition 
that results in “marked and severe functional limitations”; and 
3) the conditions(s) has lasted or is expected to last at least 1 
year or expected to result in death.[32] In most states, children 
who are Medicaid-eligible under the Aged/Blind/Disabled (ABD) 
category are the same children who qualify for Supplemental 
Security Income (SSI) under the SSA’s disabled definition. In 
Ohio and other “209b” states, the eligibility criteria for 
Medicaid are more restrictive. In these states income-eligibility level for Medicaid may be set 
lower than that required for SSI eligibility, although it also allows for “spend down” to Medicaid 
monthly income eligibility by deducting medical expenses.[33] The result is that most, but not 
all, SSI children in Ohio also qualify for Medicaid under Ohio’s ABD definition (Figure 1). 
Because Medicaid eligibility is not automatic for all SSI children in Ohio, we refer to the 
disabled children population in this research proposal as “ABD children.”      
Study Justification 
There are approximately 1.3 million children in the United States who have physical or 
behavioral health conditions resulting in significant functional impairment [1]. In Ohio, there 
are approximately 38,000 ABD children.[34] They represent 3.4% of all Ohio Medicaid children, 
yet they account for 16.2% of all spending for Ohio Medicaid children ages 0-18.[34] The 
average monthly cost per ABD Medicaid child is $1,188 compared to $205 per month for all 
other Medicaid children.[34] The involved care needs of these ABD children, along with 
concerns about health plans’ provider network adequacy, experience, and willingness to serve 
this population, have been long considered great enough to exempt them from the managed 
care enrollment required of other Medicaid-eligible children and to leave them instead in the 
traditional fee-for-service system.[2] However, faced with increased pressure to improve care 
for beneficiaries with complex needs and address budgetary concerns, several states now 
mandate managed care enrollment and/or are investing in alternative models such as ACOs for 
the pediatric ABD population.[31] The significance of this study is that it compares the 
effectiveness of care coordination for disabled children under two models of health care 
delivery—fee-for-service and ACO.   
Projected Use of Study Results 
x By heath systems (including PFK specifically and others more generally) who want to 
understand the advantages and disadvantages of an ACO model for coordinating the 
care of children with disabilities and to plan more effective approaches to care for this 
population. 
 
Figure 1:SSI vs. ABD in Ohio

5 
 
• By patients and families considering participating in an ACO—results offer them 
information for understanding the advantages and disadvantages as compared to a 
more traditional fee-for-service health plan. 
Study Design 
General Approach 
The proposed study uses a recent policy change in Ohio as a natural experiment to assess 
outcomes and experiences of ABD children and their caregivers in an ACO model of care 
compared to their previous outcomes and experiences in a traditional fee-for-service model.  
Prior to the policy change, nearly all of the 38,000 Medicaid-eligible ABD children in Ohio were 
enrolled in a traditional fee-for-service plan. After the policy change went into effect in July 
2013, all ABD children were required to enroll in managed care. In central and southeastern 
Ohio, the policy change effectively moved approximately 8,000 Medicaid-eligible ABD children 
into the Partners for Kids (PFK) ACO. 
  
PFK is the nation’s largest and oldest pediatric ACO focused solely on Medicaid-enrolled 
children. Established in 1994, PFK is a not-for-profit physician hospital organization jointly 
owned by Nationwide Children’s Hospital in Columbus, Ohio and by a 750+ physician group 
comprised of primary care physicians, pediatricians, and specialists. In 2013, Nationwide 
Children’s Hospital served 1,000,000 patient visits, 21,000 annual inpatient admissions and 68 
facilities including outpatient centers, urgent care, research, and primary care and specialty 
physician offices. As an ACO, PFK currently provides both primary and specialty care to more 
than 300,000 Medicaid children enrolled in one of 5 Medicaid managed care plans operating in 
34 counties (out of 88 total counties in Ohio) stretching from urban Columbus to rural 
Appalachia. 
Research Questions 
The proposed study will address the following research questions:  
1) Can an ACO implement a system of care coordination that improves upon what 
children with disabilities and families received before becoming part of an ACO?; and 
2) What effects do these activities have on health care and outcomes? 
Hypothesis 
Children with disabilities will have improved outcomes and experiences after becoming part of 
an ACO compared to their prior outcomes and experiences in a traditional fee-for-service 
model.  
 
 

6 
 
Study Aims 
Aim 1 
Use qualitative methods to identify stakeholder perspectives on care coordination before and 
after the transition from fee-for-service to the ACO model of care for children with disabilities. 
Aim 2 
Implement a novel caregiver and medical record abstraction tool designed to assess care 
coordination for children with special health care needs to evaluate the quality of care 
coordination provided by the ACO. 
Aim 3 
Compare healthcare quality, utilization, and other patient-centered outcomes for children with 
disabilities before and after joining the ACO and the impact of care coordination on these 
outcomes using data collected in Aims 1 and 2, along with claims and electronic health record 
data.   
Methods and Data Analysis 
Approach to Analysis: Mixed Methods 
Integrating the quantitative and qualitative analyses across aims 
A key strength of our research design is the ability to triangulate quantitative and qualitative 
approaches, thereby leveraging the strengths while minimizing the weaknesses of each.[35] 
This integration involves two approaches. First, findings from one Aim will be used to develop 
deeper insights from the other. For example, our analyses of the qualitative data, along with 
particular stories contained in that data set, may provide additional hypotheses to test using 
the quantitative data set and will be useful for developing explanations for the patterns we find 
in the quantitative analyses. Similarly, relationships observed among variables in the 
quantitative data analyses may be useful when inferring the extent to which findings from the 
qualitative analyses are likely to be generalizable.  
 
Second, we will triangulate by merging the qualitative findings with the quantitative findings to 
develop a complete, well-substantiated analysis of the comparative effectiveness of care 
coordination under the fee-for-service and ACO models. Convergence in the results from the 
qualitative and quantitative analyses will provide stronger support for comparative 
effectiveness findings, while any divergences in the results of the analyses will be useful for 
tempering interpretations of findings and guiding subsequent research efforts. For example, 
our comparison of the quality of care under the two models will integrate patient perceptions 
of quality, as identified in the focus groups and interviews, with quantitative measures from the 
claims data, such as ED readmissions. Taken together, the findings from these multiple data 
sets will provide a complex, patient-centered understanding of how quality of care differs under 
the two models. 

7 
 
Aim 1 
We will conduct key informant interviews with ACO stakeholders (Sub-aim 1a), as well as focus 
groups with patients and caregivers and interviews with caregivers (Sub-aim 1b).  Table 1 
provides an overview of Aim 1.  
Participants and Sample Size 
Table 1: Aim 1 Study Participants  
Data 
Collection 
Participants 
Interview Topics 
Payment 
Sub-aim 1a 
Interviews with 
ACO 
Stakeholders 
20 Stakeholders 
• 5 ACO Leaders  
• 5 ACO Care Coordinators  
• 5 Clinicians  
• 5 Payor Representatives  
• Care coordination before 
& after policy change 
• Delegation 
• Metrics and Evaluation  
• Hospitalization/Discharge  
• Overall Assessment 
None 
Sub-aim 1b 
Focus groups 
with Patients & 
Caregivers 
110 Individuals (across 10 focus groups) 
• 2 Caregiver Groups: Physical disability, rural area 
• 2 Caregiver Groups: Physical disability, urban area 
• 2 Caregiver Groups: Mental disability, rural area 
• 2 Caregiver Groups: Mental disability, urban area 
• 2 Patient Groups: Teenagers 
• Coordinating Care  
• Accessing Needed Care  
• Satisfaction and Quality  
• Hospitalizations 
• Additional Concerns 
• Overall Assessment 
$50 gift 
card and 
light meal 
Sub-aim 1b 
Caregiver 
Interviews 
 
30 Caregivers  
• Dependent on findings from focus groups. Will 
entail a mix of rural and urban areas and physical 
and mental disabilities 
• Coordinating Care  
• Accessing Needed Care  
• Satisfaction and Quality  
• Hospitalizations 
• Additional Concerns 
• Overall Assessment 
$50 gift 
card 
Total 
160 Stakeholders, Patients and Caregivers Study Participants  
 
Data Collection 
Sub-aim 1a: Interviews with ACO Stakeholders: We will conduct key informant interviews with 
ACO stakeholders in order to identify care coordination strategies for ABD children before and 
after the policy change. We will sample purposively from four groups: ACO leaders, care 
coordinators, clinicians, and payor representatives. Each group will provide a different 
perspective on coordination objectives and strategies. ACO leaders can speak to the goals of 
the ACO. Care coordinators and clinicians can speak to the day-to-day practices of coordinating 
care. Payor representatives can speak to the motivations to delegate care coordination to the 
ACO and ongoing efforts to monitor this delegation. We will work with the ACO to identify and 
select key informants from each of the four stakeholder groups based on their knowledge 
about and experience with care coordination related decisions and practices for ABD children. 
At least two researchers will attend each in-person interview. The role of the researchers will 
be to prompt for additional details and to take notes. Each interview will last about one hour 
and will be digitally recorded and subsequently transcribed.  
 
Sub-aim 1b: Focus groups with patients and caregivers and interviews with caregivers: We will 
conduct focus groups with patients and caregivers and interviews with caregivers in order to 
obtain their perceptions of the relative quality, value and experience of care and care 

8 
 
coordination under the two systems. We will recruit participants through the following means: 
1) distribution of flyers throughout ACO provider sites and ACO care coordinators; 2) 
recruitment email sent through Patient Advisory Panel and Voice’s For Ohio’s Children listserv; 
and 3) snowball-sampling approaches to identify additional participants based on referrals from 
other participants. To be eligible to participate, the child (as the participant or of the caregiver) 
must: 
1. Have resided continuously in the region served by the ACO, have been continuously 
enrolled in Medicaid, and have had ABD status since at least one year before the policy 
change; 
2. Be no more than 18 years of age at the time of data collection; and 
3. Have been, at the time of the policy change, at least 14 years of age for patient focus 
groups or 2 years of age for caregiver focus groups and interviews. 
These criteria ensure that participants will have had sufficient, relevant experience under both 
systems to be able to compare them.  All recruitment materials will be developed by the 
research team in consultation with Patient Advisory Panel members and will be subjected to 
review by the Institutional Review Boards at UNC-Chapel Hill, The Ohio State University, and 
Nationwide Children’s Hospital. 
 
We will conduct a total of ten 1.5-hour focus groups, each with roughly 10-12 participants. 
Under guidance from members of our Patient Advisory Panel, we have developed a strategy 
that samples based on broad categories of disability and geographic location (Table 1). This 
approach will allow us to maximize the overall variation in our sample through inter-group 
diversity, while simultaneously ensuring focused, productive discussions within each group 
through intra-group similarity. While caregivers of ABD children are often the primary point of 
contact for coordination activities, we recognize that individuals with disabilities will have 
unique perspectives. Two focus groups with disabled teenagers will therefore be conducted to 
ensure these perspectives are incorporated. All focus group discussions will be guided by 
questions we have developed with input from our Patient Advisory Panel Members (Appendix 
C). One researcher will moderate these discussions, ensuring all participants have opportunities 
to share their thoughts, including those that differ from the majority. A second researcher will 
take notes. Focus group discussions will be recorded and transcribed.  
 
Interviews with Caregivers: We will conduct 30 semi-structured interviews with caregivers of 
ABD children in order to gather in-depth, detailed accounts of their experiences with care 
coordination under both the fee-for-service and ACO models. Consistent with the logic of 
purposive sampling in qualitative studies, our goal is not to achieve statistical representation, 
but rather to ensure that our sample is comprised of richly informative cases.[35,36] We will 
therefore recruit those individuals who, by reason of their experiences, are most likely to 
provide useful insights for understanding the diverse challenges of coordinating care for ABD 
children and for evaluating the comparative effectiveness of the two models. Interviews will be 
guided by questions, refined with input from Patient Advisory Panel Members. One or two 
researchers will be involved in these interviews, prompting for additional details and to taking 
notes. Each interview will last about one hour and will be recorded and subsequently 
transcribed.  

9 
 
 
Instruments 
We have developed interview guides for ACO stakeholder interviews, caregiver interviews and 
focus groups with patients and caregivers. The ACO stakeholder interview guides include 
questions around the following domains:  Medicaid Policy change, Care Coordination, Metrics 
and Evaluations, Hospitalizations, Overall Assessment.  The caregiver interviews and focus 
group interview guides current include questions around the following domains:  Impact of 
Disability, Coordinating Care, Accessing Care, Satisfaction and Quality of Care. The caregiver and 
focus group interview guides will be vetted by our Patient Advisory Panel to make sure we are 
covering appropriate topics and utilizing clear and appropriate language in our interviews.   
Data Analysis 
We will use the same methods for analyzing all of the data gathered for Aim 1. Following 
standard qualitative coding techniques, we will code data segments within transcripts using 
labels that capture ideas contained in the data.[37,38] Related codes will then be grouped into 
themes that highlight common perceptions, ideas, or experiences across informants. We will 
follow an iterative approach to analysis that involves ongoing cycles of reading and coding 
transcripts, reviewing the literature, and discussing findings among the research team to 
identify themes. Throughout the process we will use the constant comparative method 
comparing data with data, data with codes, codes with codes, and codes with themes, in order 
to construct a detailed framework of perceptions regarding the effectiveness of care 
coordination strategies. [39,40] The research team will use the Atlas.ti software package 
(version 9.0) to facilitate the managing and coding of qualitative data. Our study investigators 
have used this approach in several previous studies.[41–44] Using all of the qualitative data 
(i.e., focus groups and ACO stakeholder and caregiver interviews), we will identify the 
advantages and disadvantages of each model and the situations under which each model is 
likely to be more or less effective based on patient and stakeholder perceptions of outcomes. 
We acknowledge that the retrospective nature of Aim 1 may affect recall. However, our intent 
will be to understand the perceptions of various stakeholder groups regarding care 
coordination under the two models, as well as, their experiences before, during and after the 
policy change. Under this aim we will investigate the extent to which participants perceive care 
coordination to have changed and their own assessments of the effects of any such changes. 
 
Aim 2 
In Aim 2, we will add to our qualitative assessment of care coordination and perceived changes 
in care coordination, using a newly developed quantitative care coordination measurement 
tool. This tool will measure experiences with and documentation of care coordination in the 
first year after the transition to the ACO. Data from this measurement will be used to describe 
the extent of care coordination, to assess whether measured care coordination is consistent 
with expectations identified in Aim 1, and to measure the relationships between care 
coordination and patient-centered utilization outcomes. 

10 
 
Caregiver Survey 
Participants & Sample Size 
Our study will implement the caregiver survey and the medical record abstraction tools in the 
PFK ABD population. Caregivers of all 8,080 newly enrolled ABD children will be invited to 
participate in the survey. We estimate that our methods will yield 2,750 caregiver survey 
respondents. The estimated 34% response rate considers a combination of those who are 
unreachable based on available contact information and those are reached but who opt out of 
participation. This sample size will allow us to estimate performance on each survey-based care 
coordination indicator within a 95% confidence interval of ±1.5 percentage points 
(conservatively assuming underlying performance of 50%). Confidence interval widths for age, 
gender, and urban/rural subpopulations will range from ± 1.8 to ±3.7. This sample size will 
allow us to identify a significant urban/rural difference (α=0.05) of 0.05 percentage points (i.e., 
0.50 vs. 0.55) with a power of 0.83. 
 
Medical Record Abstraction 
Participants & Sample Size 
We further estimate that 1,000 of those who complete the survey will consent to a review of 
their medical records that will allow data collection on medical record review based measures 
that can be linked to survey data and ultimately medical claims data. This consent rate is based 
on experience in current pilot work. This sample size will allow us to us to estimate 
performance on each medical record-based care coordination indicator within a 95% 
confidence interval of ± 2.9 percentage points (conservatively assuming underlying 
performance of 50%). Confidence interval widths for age, gender, and urban/rural 
subpopulations will range from ± 3.6 to 7.0. This sample size will allow us to identify a 
significant urban/rural difference (α=0.05) of 0.10 percentage points (i.e., 0.50 vs. 0.60) with a 
power of 0.89. 
Analysis 
We will first calculate performance on each of the 31 care coordination measures using data 
from the FECC and the care coordination medical record review. We will stratify indicator 
performance by age group, urban/rural residence, and disabling diagnosis (physical vs. mental 
diagnosis). Where appropriate, performance in our sample will be compared with pilot data 
collected by COE4CCN. We will then link data from the FECC, the medical record review, and 
the PFK claims data, at the individual level, to assess the relationship between receipt of care 
coordination elements and selected outcomes. For example, this linked database will allow us 
to test the relationship between receipt of a useful and understandable discharge summary 
following an inpatient stay and 30 day readmission or the relationship between regular 
communication with a designated care coordinator and use of the emergency department. 
Specific quality/outcomes measures to be studied will be drawn from claims and EMR-based 
CHIPRA and HEDIS measures with input from our advisory panel. We expect limited missing 
data in the survey and medical record data due to our data collection methods. Missing data 
from the survey and the medical record review will be handled using the “available case 

11 
 
analysis approach” in which all cases with complete data for a selected analysis are included in 
that analysis. 
 
The proposed analysis in Aim 2 will allow us to 1) quantitatively assess the quality and 
completeness of care coordination offered to ABD children upon roll-in to the ACO; 2) 
determine whether that quality and completeness is consistent across selected subpopulations; 
and 3) understand the relationships between elements of care coordination and patient-
centered measures of quality and outcomes.  
 
Aim 3 
In Aim 3, we will compare quality, utilization, and other patient-centered outcomes for ABD 
children before and after joining an ACO in Ohio using claims, survey and medical record data. 
Our comparative analysis includes both quantitative and qualitative measures identified 
through Aims 1 and 2. In addition, we will triangulate findings from our quantitative, qualitative 
and care coordination assessment analyses to provide a robust and multi-faceted analysis of 
the comparative effectiveness of care coordination under a fee-for-service system compared to 
an ACO model for children with disabilities.  
 
Data & Sample Size: Our primary analytic data come from Ohio Medicaid administrative claims 
for years 2010-2016. This data captures service utilization, dates of service, diagnoses and 
expenditure data paid by Ohio Medicaid for the 8,080 ABD children attributed to the ACO. 
Claims data include encounter data on patient level services provided through the ACO as well 
as those directly reimbursed through Medicaid, including all mental health services paid by 
Ohio Medicaid. 
 
Using the ABD eligibility identifier in the state Medicaid claims data, we will be able to identify 
the Medicaid beneficiary numbers and claims histories of the estimated 8,080 ABD children 
who automatically transitioned from fee-for-service to the ACO after the policy change, 
allowing for comparison of utilization and claims based quality measures prior to joining the 
ACO. For the subsets of this population who complete the survey and who consent for medical 
records review from Aim 2, these additional rich data sets will be merged with the claims data 
to test whether aspects of care coordination impact care, quality and utilization differences 
after the policy change.  
 
We do not anticipate substantial missing data issues from the Ohio Medicaid claims database. 
While the claims data will have the traditional limitations of administrative data (i.e., the 
potential for under-coding of non- diagnoses, and the lack of information on reimbursed 
services), only paid claims will be used in the analysis. Thus, payment data and other fields such 
as patient id, date of service, procedure codes required for reimbursement will be well 
populated for each claim. This sample size is more than adequately powered; subgroup 
analyses described below will be conducted only when the subgroup size is large enough to 
have adequate power for the analyses. 
 

12 
 
Outcomes Measures: We will construct quality, utilization and other patient-centered 
outcomes measures for our comparative analyses. Our proposed measures, described below, 
draw on a review of the literature and existing recommendations by the Children’s Health 
Insurance Reauthorization Act (CHIPRA), the Healthcare Effectiveness Data and Information Set 
(HEDIS), and other quality-reporting metrics designed specifically for children and children with 
complex health care needs. Measures will be specified quarterly and annually as appropriate.   
 
Administrative data-derived measures include items such as continuity of care, use of the 
emergency department, hospitalizations, and access to primary and specialty care, timely 
follow-up after initial prescriptions of ADHD medications or after psychiatric 
hospitalization.[45,46] We will incorporate recent methodological advances in operationalizing 
these measures from administrative data.[47] All study measures will be reviewed with our 
Patient Advisory Panel in order to prioritize the measures determined to be of greatest 
importance to our patient population. The Patient Advisory Panel may also propose and 
prioritize additional patient-centered outcome measures during the first two years of the 
project.   
 
Key treatment variable: The key variable of interest is the enrollment in the ACO by children 
with disabilities. This “treatment” was assigned to children seen by one of the health care 
facilities within PFK’s 34-county service area in Ohio beginning in July 2013. Data from PFK-
enrolled children prior to enrollment (July 2010 – June 2013) will serve as primary controls, 
adjusting for age and length of time on Medicaid. Children who were enrolled in Ohio Medicaid 
with ABD eligibility in other areas of the state will serve as additional potential controls using 
Medicaid claims data. We cannot use these children as primary controls due to important 
limitations in the data, namely the inability to attribute children to other ACOs outside the 34 
county PFK region. PFK’s unique arrangement with the state allows us to clearly identify the 
ACO catchment area by county and thus we can clearly attribute these patients to PFK.   
 
Patient Characteristics: Our analysis is conducted on a diverse population of children and 
adolescents. Their characteristics vary by age, gender, race, Latino ethnicity, SSI status, county 
of residence, diagnoses, procedures received, length of time on Medicaid and other patient 
characteristics as detailed in Appendix F. We will create baseline disease adjustment categories 
using the Chronic Illness and Disability Payment System, which was specifically designed for 
disabled Medicaid enrollees. [48] 
 
Methods: Following the latest guidance on PCOR methods from the PCORI Methodology 
Report, we will define the analysis sample that represents the distribution of demographic and 
conditions among ABD children in Ohio from histories available in the administrative data.[49] 
We will use approaches appropriate to PCOR including mixed qualitative and quantitative 
methodologies in a quasi-experimental study design for this Aim.  
 
Quantitative Analysis: Using the interrupted time series analysis for statistical models, we will 
estimate the effect of switching from FFS to ACO while controlling for secular trends. This 
method examines both the change in the average level of each outcome in the post period as 

13 
 
compared to the pre-ACO period, as well as changes in the trends in each outcome. For 
example, an interrupted time series analysis can determine whether access to outpatient visits 
with specialists increases or decreases after the children are enrolled in PFK as well as whether 
changes in the rate of access over time are greater or lower as compared with the pre-ACO 
period. Outcome models will be run separately on each outcome using generalized estimating 
equations with appropriate distributional assumptions for each type of outcome variable (e.g., 
binary, continuous). The key variables will be an indicator of the ACO enrollment period and its 
interaction with time trends. 
 
While we believe this is the strongest study design for this Aim given the available data, the lack 
of a control group cannot rule out other explanations for differences in outcomes over time. 
Therefore, we will carefully consider alternative explanations to other trends with our Patient 
Advisory Panel and other key experts once we obtain preliminary analytic results. We will 
additionally examine the potential for controls from children in other regions of Ohio for a 
case/control analyses. Although ACO assignment was mandatory and assigned based only on 
region and thus may not suffer from self-selection bias, if we are able to incorporate data from 
children in other regions, we will carefully examine the baseline characteristics, utilization 
during the pre-treatment period and risk factors between ACO and other ABD children in Ohio 
to ensure balance in covariates. Incorporating control data from Medicaid claims for children in 
states other than Ohio is not plausible, given that these data are available with a three-year lag 
and thus post-period analysis would not be possible during our study period.  
 
Sub-group analysis: Although our primary goal is to estimate the effect of ACO enrollment on 
vulnerable ABD children overall, we will also estimate the heterogeneity of the ACO treatment 
effect in populations defined by age, race/ethnicity, gender, care coordination metrics and 
diagnostic group to the extent allowable by sample size. The specific subgroups will be 
identified in advance collaboratively with our stakeholders. This sub- analysis will control for 
the health status and disease progress of a patient during the 6-year period. 
 
We will also look at the impact of specific elements of care coordination provided on a sub-
group of our population using information collected in our caregiver survey and medical record 
review. For example, as noted above, one of our potential outcomes variables is use of the 
emergency department. One potential research question is whether emergency department 
use is lower in children whose caregivers reported receipt of an understandable visit summary 
after their last outpatient visit. 
Data Safety, Monitoring, and Handling 
Aim 1 
Interview data collected from participants will be identified only by an assigned participant ID 
number, not by name. The data confidentiality and informed consent process will be explained 
by the investigator prior to the start of the interview and before the recording begins. 

14 
 
Participants will have the option to not respond to any question and to not have their 
interviews recorded. 
Similarly, for focus groups, participants will be identified only by number, not by name. The 
data confidentiality and informed consent process will be explained by the moderator prior to 
the start of the discussion, and before the recording begins. Participants will have the option to 
not respond to any question posed; if they do not wish to be recorded, they may elect to leave 
the discussion group. 
Digital recordings and transcriptions of interview and focus group sessions, and any documents 
collected from ACO stakeholders will remain in the possession of the study investigators at all 
times and will be reviewed in seclusion. These will all be secured in a locked office at all times 
and stored on a password-protected computer. Upon completion of the study, all electronic 
and hard copy data will remain in password-protected files and reside with the principal 
investigator; these will be destroyed after five years. 
Any forms and incentive receipt lists, which include patient names and study ID numbers, will 
be stored in locked file cabinets in a locked file room in a secure building 
For both focus groups and interviews, participants’ comments will never be quoted or reported 
in ways that disclose their identities. For medical record and Medicaid claims data, findings will 
only be reported in aggregate or summary form. 
Aim 2 
Data from the telephone survey will be entered into an electronic database as it is collected. 
Once surveying is complete, a formatted and labeled SAS data file will be provided to the PI 
using a HIPAA compliant data transfer protocol. The data from the medical record abstraction 
will be directly entered into a Redcap database, which will also export data into a SAS format. 
Ultimately, linked survey, medical records, and claims data will be housed on a secured server 
at Nationwide Children’s Hospital. Access to the data on the server will be limited to the 
project’s investigators and programmer. 
Aim 3 
NCH will provide UNC with de-identified data from the FECC survey, medical records, and 
medical claims. 
Handling Missing Data – PCORI Methodology Standards MD-1-5 
Describe methods to prevent and monitor missing data (MD-1) 
The underlying population for this study is all children up to age 18 enrolled in the Partners for 
Kids (PFK) on May 30th 2015 who had been previously enrolled in fee-for-service (FFS) Medicaid 
and transitioned into PFK as a result of the ABD policy change. The primary analytic data comes 
from Ohio Medicaid administrative claims. Claims data have different missing data issues than 
do other primary or even secondary data sets on several dimensions. First, many of the fields 
are mandatory, which means fields such as procedure code and Medicaid eligibility status will 

15 
 
be 100% complete. Second, although claims data have the traditional limitations of any 
administrative data, including the lack of information of services paid fully out-of-pocket or 
financed through Medicare, only services provided through Medicaid will be used in the 
analysis, which will take a Medicaid/agency perspective. This means that services and 
expenditures will not aim to be a complete record, but those covered through the Medicaid 
program.  We will clearly indicate the scope of analysis in publications. For the subsets of claims 
data merged with survey and medical records, we do not anticipate missing data problems for 
the same reasons above. 
 
The estimated size of this population is 8,000. Because of the qualitative nature of Aim 1 we do 
not propose a missing data plan. The following assumptions have been made regarding rates of 
participation from this population for Aims 2 and 3: 
Aim 2 – Caregiver Survey 
We expect 34% percent of the eligible caregivers to complete the caregiver survey. This is based 
on assumptions from previous studies with this population regarding degree of incorrect 
addresses for the opt-out mailing, opt outs, missing/non-functional telephone numbers, and 
refusal when contacted. 
• Prevention of missingness– We will use alternative sources of contact information (e.g. 
EHR records, internet look-up sites) for those families with missing telephone numbers. 
• Monitoring of missingness – we will record and report the following statistics: % of 
invitation letters returned by the post office, % opting out, % bad telephone numbers 
that cannot be found after research, and % refusal by reached families. For each of 
these percentages, we will describe the lost respondents by gender, age, urban/rural 
residence, and diagnosis category using PFK data (note: race is not reliably available on 
the PFK data). 
 
We expect 35% percent of those completing the telephone survey to consent to medical record 
abstraction.  
• Prevention of missingness – The script requesting medical record access will clearly 
explain the reason for access and the processes for maintaining confidentiality. 
• Monitoring of missingness– We will describe the lost respondents by gender, age, 
urban/rural residence, and diagnosis category using claims data. We will also test the 
relationship between perceptions of care coordination and consent to medical record 
abstraction using data from the telephone survey. 
 
A consort-style diagram will be used to illustrate potential participant loss or refusal at each 
stage of the study (MD-4). 
Aim 3 – Utilization Analysis 
Over 95% of the population will be included in claims data analysis looking at overall pre-/post- 
policy change utilization analysis. The primary risk of missingness for this analysis is failure of 
successful linking between the FFS (pre) and PFK (post) data. After acquisition of the data, we 
will first assess the data quality. Like the traditional Medicaid plan, PFK reimburses physicians 

16 
 
on the basis of services provided (fee-for-service), therefore we expect that the payment data 
and other fields such as patient id, date of service, and procedure codes required for 
reimbursement will be populated for each claim billed to Medicaid/PFK. The demographic files 
(e.g. gender, race) may also have missing data, which is anticipated to be proportionally small, 
based on the team’s prior experience with claims.  
• Prevention of missingness – We will investigate each non-match. When there is no 
matching Medicaid ID across the two databases, we will directly check the record to 
identify potential transposition or typographical errors. If no such error can be found, 
we will attempt to use probability matching using date of birth, zip code of residence, 
gender, and soundex last name to link records. 
• Monitoring of missingness– We will calculate and report the final “no-match” rate 
between PFK and state FFS data. 
 
Statistical methods and sensitivity analysis to handle missing data (MD2, MD3, MD5) 
Aim 2 – Caregiver Survey 
Methods for statistical handling of missing responses on the Family Experiences of Care 
Coordination survey are documented in the survey’s scoring guide. We will use these methods 
to assure that our results are consistent with those being generated in other implementations 
of the survey. Missing data will not be imputed, however, we will use sensitivity analysis to 
estimate the effect of differential non-response by age, gender, urban/rural residence, and 
diagnosis category. We do not hypothesize differential missingness. 
Aim 3 - Utilization Analysis 
The pre-/post-policy change utilization analysis will be performed on all individuals where we 
are able to link FFS and PFK claims. We expect a low non-link rate. However, to estimate the 
impact of the missing data, we will conduct sensitivity analysis by testing change in setting-
specific utilization if all the unlinked cases were at the 90th percentile of their 
age/gender/diagnosis category group and if all unlinked cases were at the 10th percentile of 
their age/gender/diagnosis category group. 
 
For time invariant missing variables in the FFS or PFK data, such as gender or race, we will carry 
the last observation forward, assuming gender and race is unchanged for each individual. In the 
rare case of substantial amount of other types of missing data in the claims, we will employ 
Multiple Imputation (MI) methods to generate imputations for the missing values, from 
predicted probabilities estimated based on the complete data (observed and missing data).   
  
Procedures 
Aim 1 
In all aspects of Aim 1, we will use purposive sampling, attempting to get a mix of participants 
and to maximize the diversity of our sample in terms of perspectives that we expect to be 

17 
 
relevant to the coordination of care for children with disabilities. For interviews with ACO 
stakeholders, we will sample purposively from four groups: ACO leaders, care coordinators, 
clinicians, and payor representatives. Each group will provide a different perspective on 
coordination objectives and strategies. For focus groups with youths and caregivers and 
interviews with caregivers, we will sample based on broad categories of condition and 
geographic location (rural v. urban). This approach will allow us to maximize the overall 
variation in our sample through inter-group diversity, while simultaneously ensuring focused, 
productive discussions within each group through intra-group similarity. Finally, for the 
interviews with Caregivers, we intend to select 30 parents, as this number provides a large 
enough sample to gather a rich mix of different stories and experiences in rural and urban 
settings and from parents of children with different types of disabilities. 
 
To be eligible to participate in ACO stakeholder interviews, participants must be involved with 
the ACO as either an ACO leader, a care coordinator or clinician, or as a representative of a 
payor organization (i.e., insurer). To be eligible to participate in focus groups or caregiver 
interviews, the child (as the participant or of the caregiver) must: 
1. have resided continuously in the region served by the ACO, have been continuously 
enrolled in Medicaid, and have had ABD status (i.e., Medicaid disability status) since at 
least one year before the policy change; 
2. be no more than 18 years of age at the time of data collection; and 
3. have been, at the time of the policy change, at least 14 years of age for youth focus 
groups or 2 years of age for caregiver focus groups and interviews.  
Children under the age of 15 will be excluded on the grounds that they may not be able to 
recall with sufficient detail their experiences under the previous Fee-for-service system. These 
criteria ensure that participants will have had sufficient, relevant experience under both 
systems to be able to compare them. 
 
Our recruitment strategies vary by sub-aim. For sub-aim 1a (Interviews with ACO Stakeholders) 
we will work with the ACO to identify and select key informants from each of the four 
stakeholder groups based on their knowledge about and experience with care coordination 
related decisions and practices for ABD children. Research staff already have relationships with 
key leaders at the Partners For Kids ACO. Researchers will use these individuals both as 
participants and also to identify other key stakeholders within the Partners For Kids 
organization and at the payor organizations with which Partners For Kids deals. The researchers 
will send information about the study along with an invitation to participate to these individuals 
via email. ACO stakeholders will not be compensated for participation. 
 
For sub-aim 1b (Focus groups with youths and caregivers and interviews with caregivers) we 
will recruit participants through the following means: 
1. distribution of flyers throughout ACO provider sites and by ACO care coordinators who 
will distribute flyers to all of their patients/caregivers with whom they interact during 
the study period;  
2. recruitment email sent through Voice’s For Ohio’s Children listserv; and  

18 
 
3. snowball-sampling approaches to identify additional participants based on referrals 
from other participants. 
 
Caregivers and patients (i.e., youths) who participate in focus groups will receive a $50 gift card 
and light meal for participation. Caregivers who participate in interviews will receive a $50 gift 
card. Compensation will not be pro-rated. If participants choose to discontinue participation 
once the interview or focus group has started, they will still be compensated the full amount. 
 
Participants in the ACO stakeholder key informant interviews, caregiver interviews, and 
caregiver or youth focus groups will be provided a fact sheet with details about the study at the 
start of the interview. All participants will be told that they may decline to participate, decline 
to be recorded, refuse to answer any question, or discontinue participation at any time. While 
many of the ACO key informants will be employees of the ACO, we will emphasize that their 
participation is completely voluntary and will in no way affect their employment with the ACO 
or Nationwide Children’s Hospital. Focus group participants and caregiver interviewees may be 
patients (or parents of patients) of providers associated with the organization being studied; 
thus, we will emphasize that their participation is completely voluntary and in no way will affect 
their ability to obtain care, their relationship with Partners For Kids, Nationwide Children’s 
Hospital or their interaction with their doctor(s). The PI or study investigator will review the fact 
sheet with participants, ask for and address any questions they may have, and then request 
verbal informed consent/assent in all cases EXCEPT for the youth focus groups, in which case 
written consent will be requested. If the participant provides consent/assent, then the study 
investigator will proceed with the interview or focus group using the interview or focus group 
guide. Additionally, in the case of teenagers participating in the youth focus groups, we will 
obtain written informed assent from the teenager participant. 
 
All interviews and focus groups will follow the list of questions and topics in the respective 
guides prepared by research staff with consultation from the patient advisory panel. The 
interviews and focus groups are semi-structured in the sense that additional, follow-up 
questions will be asked to probe comments made by participants. Interviews with ACO 
stakeholders and with caregivers will last about 1 hour. Focus groups will last about one and 
one half hours. All interviews and focus groups will be digitally recorded and subsequently 
transcribed. 
 
STEPS – Youth Focus Groups (see IRB) 
1. Post flyers, send recruitment email (list document names) 
2. Youth call 888-218-1040 number or send email to 
caregiverinterview@nationwidechildrens.org– – can leave a confidential message (write 
out confidential message) 
3. Ascertain eligibility (form name: )obtain verbal assent during telephone recruitment 
4. In lieu of signed parental permission, we will provide study information to parents over 
the phone and gain parental consent for participation and receiving a gift card. . 
Specifically, when the youth call about the study, we will ask to speak to a parent and 
describe the study to the parent. If the parent approves of the youth's participation, we 

19 
 
will continue with enrolling the youth into the study. If the parent does not wish for the 
youth to participate, we will thank the parent for his/her time and conclude the call. 
5. If a parent is not available over the phone, we will ask the youth for an address and mail 
a fact sheet to the parent prior to the focus group. The fact sheet has clear information 
about whom to call should the parent not want their child to participate. The youth will 
need to have the parent call back or come with the youth to the focus group to obtain 
verbal or signed parental permission 
6. Mail or email directions to the youth. 
7. ARRIVING AT FOCUS GROUP. Signage will be placed to guide participants to the proper 
location.  At the focus group we will provide the participants with an additional copy of 
the fact sheet and then written assent when the focus group convenes. 
 
 
STEPS – Parent/Caregiver Focus Groups (see IRB) 
1. Caregivers call 888-218-1040 number or send email to 
caregiverinterview@nationwidechildrens.org– can leave a confidential message (write 
out confidential message) 
2. Ascertain eligibility (form name: ) obtain verbal consent during telephone recruitment 
3. Mail or email directions to the caregiver. 
4. ARRIVING AT FOCUS GROUP. Signage will be placed to guide participants to the proper 
location.  At the focus group we will provide the participants with an additional copy of 
the fact sheet and then written assent when the focus group convenes. 
 
STEPS – Parent/Caregiver Interviews (see IRB) 
1. Post flyers, send recruitment email 
2. Caregivers call 888-218-1040 number or send email to 
caregiverinterview@nationwidechildrens.org– can leave a confidential message (write 
out confidential message) 
3. Ascertain eligibility (form name: )obtain verbal assent during telephone recruitment 
4. Mail or email directions to the participant. 
5. ARRIVING AT INTERVIEW. Signage will be placed to guide participants to the proper 
location. At the interview provide participants with an additional copy of the fact sheet. 
We will use the same methods for analyzing all of the data gathered for Aim 1. Following 
standard qualitative coding techniques, we will code data segments within transcripts using 
labels that capture ideas contained in the data. Related codes will then be grouped into themes 
that highlight common perceptions, ideas, or experiences across informants. We will follow an 
iterative approach to analysis that involves ongoing cycles of reading and coding transcripts, 
reviewing the literature, and discussing findings among the research team to identify themes. 
Throughout the process we will use the constant comparative method comparing data with 
data, data with codes, codes with codes, and codes with themes, in order to construct a 
detailed framework of perceptions regarding the effectiveness of care coordination strategies.  

20 
 
The research team will use the Atlas.ti software package (version 9.0) to facilitate the managing 
and coding of qualitative data. Our study investigators have used this approach in several 
previous studies. Using all of the qualitative data (i.e., focus groups and ACO stakeholder and 
caregiver interviews), we will identify the advantages and disadvantages of each model and the 
situations under which each model is likely to be more or less effective based on patient and 
stakeholder perceptions of outcomes. We acknowledge that the retrospective nature of Aim 1 
may affect recall. However, our intent will be to understand the perceptions of various 
stakeholder groups regarding care coordination under the two models, as well as, their 
experiences before, during and after the policy change. Under this aim we will investigate the 
extent to which participants perceive care coordination to have changed and their own 
assessments of the effects of any such changes. 
 
Interview data collected from ACO stakeholder participants will be identified only by position 
title, not by name. Similarly, for interviews and focus groups with caregivers and youths, 
participants will be identified only by number, not by name. Participants’ comments will not be 
quoted in such a way as to disclose their identity.  All data will be reported either in aggregate, 
summary, or individually de-identified form. 
 
Aim 2 
Caregiver Survey Methodology 
The FECC will be administered by the Carolina Survey Research Lab (CSRL) at the University of 
North Carolina at Chapel Hill. Survey steps are outlined below: 
• The ACO (PFK) mails letters to potentially eligible enrollees describing the study and offering 
them the option of opting out of the sampling frame. Contact information for those who do 
not opt out will be sent to CSRL for survey contact. 
• CSRL will send letters to all non-opt out families. The letter will include thanks for 
willingness to participate, telephone survey logistics, and a web address that can be used to 
complete the survey on their own. We expect up to five percent of respondents to use the 
web-based survey. 
• Trained interviewers will make up to 10 telephone attempts for each caregiver who does 
not complete the online survey. When telephone numbers are found to be non-functional, 
research staff will make reasonable efforts to find working numbers through use of Lexus-
Nexus and through working with PFK staff with access to electronic medical record (EMR) 
information that may offer updated contact information. 
• Surveys will be conducted using a computer-assisted telephone interviewing (CATI) system 
with interviewer performance monitored through review of randomly selected taped 
interviews. 
• Prior to the close of the interview, the interviewer will request verbal consent for a review 
of care coordination documentation in their medical records and for linking of their survey 
responses to medical record and claims data.  
• All individuals who complete the interview will be mailed a $20 gift card in appreciation of 
their participation. 

21 
 
Key Decisions 
o We will keep the survey wording the same because it has been field tested and 
underwent cognitive development testing. 
 It is not necessary to allow for N/A (for e.g. 5E – needing equipment, in all 
the piloted work this was never an issue). 
 Refusals or don’t know – they may not be read, but interviewer can fill it in. If 
subject says they don’t know, record in notes and leave unanswered or say 
respondent refused. For programming purposes, refuse/don’t know is a 
(non-read) option. Email-based surveys can allow blanks (in fact are 
mandated to allow). 
o Web survey 
 We will pilot the web survey. 
•  CSRL will do in-house.  
• We will also pilot with Patient Advisory Panel prior to April 6th patient 
advisory panel meeting. 
 We will work on web survey first to make sure it is available for pilot testing, 
and then focus on the phone survey. 
 CSRL will mail the follow-up letter to non-opt-out families with a web link 
and code. Participants will have one month to answer. After the month, CSRL 
will discontinue access to the web and then will know which part of the 
sample remaining to be called.  
o Eligibility/Survey Administration 
 NCH will provide a summary of demographic data on opt-outs.  
 Families with more than 2 eligible patients – ask about the patient (i.e. child) 
for whom the interviewer is calling only. 
 Interviewer will ascertain which caregiver is the one who obtains health care/ 
the most involved in going to health care visits with the child and interview 
this person.  
 At end of survey, interviewer will confirm to whom the check should be 
mailed and confirm mailing address – participant can choose any person to 
send check to. 
 We will add ‘Yes I agree, No I don’t agree’ for consent box at the end of 
survey – this will serve as documentation that interviewer obtained verbal 
consent. 
o Reporting 
 CSRL will provide weekly reports on calls that will include such information 
as: bad numbers, number called, number ineligible, those we haven’t 
screened, screened but declined, and completed, and medical record 
consents. May also include how long the interview has taken and profiling.  
 
Timeframe 
March 1 
UNC provide CSRL with all IRB-approved materials, including consent 

22 
 
materials, screener, and the survey in both English and Spanish. 
March 27 
CSRL send web-based survey link for pilot testing with the patient advisory 
panel. 
March 27-
April 5 
Patient advisory panel members pilot the web-based survey. 
 
April 6 
Quarterly Advisory Panel Meeting – investigators solicit PAP input including 
such things as need for more instructions, navigation issues, using the 
interface (do links work, etc.), is survey user-friendly. 
April 10 
UNC provide feedback about the web-based survey to CSRL 
May 1 
PFK mail out opt-out letters. 
May 31 
Preliminary demographics due to PCORI (prepared by NCH). 
June 9 
End of 40-day period. 
June 23 
NCH send eligible sample to CSRL.  
July 1 
Begin web survey. CSRL send follow-up letters to eligible sample with code 
and web address. 
July 31 
End date for web survey. 
August 1 
Begin telephone survey. 
October 31 
End date for telephone survey. 
Nov 30 
CSRL send final dataset to UNC. 
 
Medical Record Review 
Based on our pilot work, we expect that 1,000 caregivers who complete the phone survey will 
also provide consent for abstraction of their medical records for MR based care coordination 
measures. Data for Medical record-based care coordination measures will be collected 
primarily through review of the Epic EMR at Nationwide Children’s Hospital (NCH) conducted by 
an experienced, trained NCH abstractor using a field-tested data abstraction tool. The 
integrated EMR at NCH allows review of NCH inpatient visits and primary and specialty care 
services provided by physicians in the NCH system. We estimate 50% of ABD children enrolled 
in PFK were seen in the NCH system in the past year. If a participant lists a main provider 
outside of the NCH system, a copy of the relevant medical records will be requested from that 
provider for review. The first 20 records will be re-reviewed by a second abstractor for quality 
control and training. 
Aim 3 
Assumptions 
- 
PFK data and claims data are housed on servers at NCH. 

23 
 
- 
NCH has permission from PFK to use data. 
- 
NCH has permission from the state to use data. 
- 
NCH will provide a limited de-identified dataset at the person-month level. 
- 
Data will be provided in SAS and STATA format. 
- 
Data will be available for download from a secure server using a double-password 
system. 
- 
Deena, Marisa, Wendy, Paula, and the NCH programmer will have access to the data. 
 
Steps for Data Preparation and Access 
1. Wendy and Marisa will develop a programming guide for the programmer.  
2. NCH Programmer links PFK and state data (EHR data, claims data) to study data 
(caregiver data) and strips identifiers. 
3. NCH Programmer creates an analytic file. 
4. NCH Programmer runs frequencies. 
5. NCH Programmer makes de-identified dataset available in both SAS and Stata formats. 
6. NCH will notify team that data are available for download. 
7. MD/WX/PS will log on to server using double-password system (VPN-like system). 
8. MD/WX/PS will download and house data on secure site at UNC & OSU. 
Dissemination Plan 
The Patient Advisory Panel will assist us in developing our final dissemination plan. The patient-
centered quality of the research will ensure that the findings disseminated to policy-makers and 
providers also emphasize patients and caregivers, alerting them to the importance of these 
groups in their decision-making going forward. The results of our research might be 
disseminated through various stakeholder groups, including families of disabled children, child 
disability advocacy groups, state Medicaid agencies, policy makers and ACO and health system 
leaders. In addition to the dissemination activities included in our proposal, the results of this 
research could be disseminated through stakeholder newsletters, webinars, websites, patient 
or community conferences, news media outlets and issue briefs. Target stakeholder meetings 
and conferences will include the AcademyHealth Annual Research Meeting and National Health 
Policy Conference and the Pediatric Academic Societies/Ambulatory Pediatrics Association 
Complex Care Special Interest group meeting, Target journal for publication will include Health 
Affairs and Pediatrics. 
   
The results of this research have real potential to be implemented in other settings, namely in 
other ACOs. In the wake of the Affordable Care Act, ACOs are developing across the country. 
Medicare ACOs have largely dominated the ACO market, however, there is increasing interest 
and growth in Medicaid and pediatric ACOs. Our research involves one of the most highly 
regarded pediatric ACOs in the country and pediatric ACOs across the state of Ohio and 
elsewhere are likely to look to results from our research to help guide their care coordination 
efforts for ABD children. 

24 
 
Collaborating Organizations, Personnel, Project Roles 
University of North Carolina at Chapel Hill 
Paula Song, Ph.D, Associate Professor, will serve as Principal Investigator. As such she will lead 
the project’s efforts and assumes overall responsibility for completing project milestones 
according to the PCORI contract.  
 
Marisa Domino, Ph.D., Professor, will serve as Co-Investigator with primary responsibility for 
Aim 3. She will work together with Dr. Xu of OSU to conduct quantitative comparative analyses 
using survey, medical record, and claims data. She will also contribute to the quantitative 
analyses under Aim 2 and the cross-aim analyses under Aim 3.   
 
Renée Ferrari, Ph.D, Research Associate, will serve as Project Manager. She will assume 
responsibility for day to day management of tasks and operations associated with the study, 
including IRB submission, study protocol development, reporting, and ensuring that project 
milestones are met and on time. Dr. Ferrari will also participate in data analysis. 
 
Ohio State University 
Brian Hilligoss, Ph.D., Assistant Professor, will be Principal Investigator of The Ohio State 
University subcontract and a Co-Investigator on the overall study. He will lead the OSU team 
and assume responsibility for assuring that all staff are performing the required tasks and that 
all reports are submitted in a timely manner. He will serve as the primary link between the OSU 
team and the project PI. He will lead the qualitative data collection and analyses under Aim 1 
and assist in the cross-aim analyses under Aim 3. He will hire and supervise a student employee 
who will handle functional tasks relating to the focus groups and other tasks as needed.  
 
Sandra Tanenbaum, Ph.D., Professor, will be a Co-Investigator on the overall study.  She will 
serve as the primary liaison between the overall study research team and the Patient Advisory 
Panel.  In addition, she will contribute to the qualitative data collection and analyses under Aim 
1 and the cross-aim analyses under Aim 3. 
 
Wendy Yi Xu, Ph.D., Assistant Professor, will be a Co-Investigator on the overall study. She will 
conduct quantitative comparative analyses under Aim 3. She will also contribute to the 
quantitative analyses under Aim 2 and the cross-aim analyses under Aim 3.   
 
A to be named hourly OSU student will be hired to assist with the focus group onsite logistics 
and to perform other functions as directed by the Principal Investigator. 
NCH/PFK 
Deena Chisolm, Ph.D., Nationwide Children’s Hospital and Partners for Kids will have overall 
responsibility for activities associated with Aim 2. For recruitment, NCM/PFK responsibilities 
include identifying the eligible population within their patient population, mailing opt-out 
letters to the eligible persons notifying them about the study, and providing the Carolina Survey 

25 
 
Research Lab (CSRL) with contact information for persons who did not opt-out of the study. 
NCM/PFK is responsible for providing de-identified medical record data to the study team for 
analysis. Dr. Chisolm at NCM will also participate in other study activities of data analysis, 
manuscript development, and study dissemination. 
 
Madhurima Sarkar, Ph.D., Research Associate, will serve as Project Manager at NCH/PFK for 
activities associated Aim 2. She will be responsible for IRB maintenance, project 
documentation, timeline management, and other administrative activities. 
 
Karen Leonhart, B.S, Senior Research Associate/Medical Record Abstractor will conduct the care 
coordination record reviews (medical record abstraction) using the NCH electronic medical 
record system.  
 
To Be Named, Programmer/Analyst, who will be responsible for data management and analysis 
for Aims 2 and 3. 
 
Gary Norwitz, MD, Associate Medical Director for Utilization Management at Nationwide 
Children’s Hospital, will provide physician perspectives on care coordination-related project 
design, implementation, and analysis.  
 
Joshua Nowack, MHA, Manager of Care Coordination at Partners for Kids, will serve as the 
liaison between the grant team and Partners for Kids. That role will include assistance with 
scheduling interviews for Aim 1 and coordinating data sharing issues. 
 
Patient Advisory Panel – Voices for Ohio’s Children 
Sandy Oxley, Chief Executive Officer, Voices for Ohio’s Children (Voices), will serve as co-
investigator with primary responsibility for leading the Patient Advisory Panel.  
 
Thomas Scheid, MA, Health Policy Consultant, Voices for Ohio’s Children in Columbus, Ohio will 
serve as co-investigator with primary responsibility for leading the Patient Advisory Panel.  
 
Ms. Oxley and Mr. Scheid will identify and invite representatives to the patient advisory panel. 
They will host quarterly Patient Advisory Panel meetings. Ms. Oxley and Mr. Scheid will also 
assist in communicating information about focus groups, caregiver interviews and surveys by 
distributing recruitment information through their email listserv. They will assist with 
dissemination efforts by inviting study investigators to present study findings at their annual 
conference and highlighting study results in Voices newsletters and related publications.  
 
Patient Advisory Panel Members – see complete list in Attachment A. 
Seattle Children’s Research Institute 
Dr. Mangione-Smith, Ph.D, developed the caregiver survey that will be implemented as part of 
Aim 2, and as such she will provide expertise on the survey administration, analysis, and 

26 
 
interpretation. Dr. Mangione-Smith will also participate in manuscript development and study 
dissemination activities. 
 
Carolina Survey Research Lab 
Robert Agans, Ph. D, Clinical Associate Professor and Co-Director of the CSRL, assumes overall 
responsibility for the caregiver survey administration. Dr. Agans will oversee the survey 
methodology, participate in determining the sampling frame.  
 
Anna Hoffmeyer, B.A., Operations Manager of CSRL, will be responsible for the caregiver survey 
administration and training and oversight of telephone interviewers.  
 
To Be Named, Telephone Interviewers will conduct the caregiver telephone survey. 
 
 

27 
 
References 
1.  
Social Security Administration. SSI Annual Statistical Report, 2012. Available from: 
http://www.ssa.gov/policy/docs/statcomps/ssi_asr/2012/ssi_asr12.pdf 
2.  
Kaiser Family Foundation. People with Disabilities and Medicaid Managed Care: Key 
Issues to Consider. [Last accessed 2013 Oct 25]. Available from: http://kff.org/medicaid/issue-
brief/people-with-disabilities-and-medicaid-managed-care/ 
3.  
Boyle CA, Boulet S, Schieve LA, Cohen RA, Blumberg SJ, Yeargin-Allsopp M, Visser S, 
Kogan MD. Trends in the Prevalence of Developmental Disabilities in US Children, 1997–2008. 
Pediatrics. 2011 Jun 1; 127(6):1034–1042. PMID: 21606152 
4.  
Centers for Disease Control.  Developmental Disabilities, Facts - NCBDDD. [Last accessed 
2013 Oct 31]. Available from: 
http://www.cdc.gov/ncbddd/developmentaldisabilities/facts.html 
5.  
Aday LA. At Risk in America: The Health and Health Care Needs of Vulnerable 
Populations in the United States. John Wiley & Sons; 2002.  
6.  
Perrin JM, Bloom SR, Gortmaker SL. The Increase of Childhood Chronic Conditions in the 
United States. JAMA. 2007 Jun 27; 297(24):2755–2759.  
7.  
Christensen D, Van Naarden Braun K, Doernberg NS, Maenner MJ, Arneson CL, Durkin 
MS, Benedict RE, Kirby RS, Wingate MS, Fitzgerald R, Yeargin-Allsopp M. Prevalence of Cerebral 
Palsy, Co-occurring Autism Spectrum Disorders, and Motor Functioning - Autism and 
Developmental Disabilities Monitoring Network, USA, 2008. Dev Med Child Neurol. 2014 Jan; 
56(1):59–65. PMID: 24117446 
8.  
Simon TD, Berry J, Feudtner C, Stone BL, Sheng X, Bratton SL, Dean JM, Srivastava R. 
Children With Complex Chronic Conditions in Inpatient Hospital Settings in the United States. 
Pediatrics. 2010 Oct 1; 126(4): 647–655. PMID: 20855394 
9.  
Goudie A, Havercamp S, Ranbom L, Jamieson B. Caring for Children with Disabilities in 
Ohio: The Impact on Families [Last accessed 2014 January 6]. Available from: 
http://www.ddc.ohio.gov/pub/OHFamImpStudyWhitePaper-FINAL.pdf 
10.  
Newacheck PW, Strickland B, Shonkoff JP, Perrin JM, McPherson M, McManus M, 
Lauver C, Fox H, Arango P. An Epidemiologic Profile of Children With Special Health Care Needs. 
Pediatrics. 1998 Jul 1;102(1):117–123. PMID: 10617723 
11.  
Anderson D, Dumont S, Jacobs P, Azzaria L. The Personal Costs of Caring for a Child with 
a Disability: A Review of the Literature. Public Heal Reports. Wash DC 1974. 2007 Feb;122(1):3–
16. PMID: 17236603 
12.  
Parish SL, Shattuck PT, Rose RA. Financial Burden of Raising CSHCN: Association With 
State Policy Choices. PEDIATRICS. 2009 Nov 30; 124(Supplement): S435–S442.  
13.  
Goudie A, Havercamp S, Jamieson B, Sahr T. Assessing Functional Impairment in Siblings 
Living With Children With Disability. Pediatrics. 2013; 132(2): e476-83. PMID: 23897909 
14.  
Corman H, Noonan K, Reichman NE. Mother’s Labor Supply in Fragile Families: The Role 
of Child Health. East Econ J. 2005; 31(4): 601–616.  
15.  
Reichman NE, Corman H, Noonan K. Impact of Child Disability on the Family. Matern 
Child Health J. 2008; 12(6): 679 – 683.  
16.  
Powers ET. Children’s Health and Maternal Work Activity: Estimates under Alternative 
Disability Definitions. J Hum Resour. 2003; 38(3): 522-556.  

28 
 
17.  
Thyen U, Kuhlthau K, Perrin JM. Employment, Child Care, and Mental Health of Mothers 
Caring For Children Assisted by Technology. Pediatrics. 1999; 103(6 Pt 1):1235–1242. PMID: 
10353935 
18.  
Stabile M, Allin S. The economic costs of childhood disability. Future Child Cent Future 
Child David Lucile Packard Found. 2012; 22(1): 65–96. PMID: 22550686 
19.  
Burton P, Lethbridge L, Phipps S. Children with disabilities and chronic conditions and 
longer-term parental health. J Socio-Econ. 2008 Jun;37(3): 1168–1186.  
20.  
Kuhlthau K, Kahn R, Hill KS, Gnanasekaran S, Ettner SL. The Well-Being of Parental 
Caregivers of Children with Activity Limitations. Matern Child Health J. 2010 Mar 1;14(2): 155–
163.  
21.  
Quint RD, Chesterman E, Crain LS, Winkleby M, Boyce W. Home care for ventilator-
dependent children: Psychosocial impact on the family. Am J Dis Child. 1990 Nov 1; 144(11): 
1238–1241.  
22.  
Emerson E. Mothers of children and adolescents with intellectual disability: social and 
economic situation, mental health status, and the self-assessed social and psychological impact 
of the child’s difficulties. J Intellect Disabil Res. 2003; 47(4-5):385–399.  
23.  
Centers for Medicare & Medicaid Services, Baltimore MS 7500 SB, Usa M. Overview. 
2013 [Last accessed 2013 Oct 2]. Available from: http://www.cms.gov/Medicare/Medicare-Fee-
for-Service-Payment/ACO/index.html?redirect=/aco/ 
24.  
McClellan M, McKethan AN, Lewis JL, Roski J, Fisher ES. A National Strategy To Put 
Accountable Care Into Practice. Health Aff (Millwood). 2010 May 1; 29(5): 982–990. PMID: 
20439895 
25.  
Press MJ, Michelow MD, Macphail LH. Care Coordination in Accountable Care 
Organizations: Moving Beyond Structure and Incentives. Am J Manag Care. 2012 Dec; 18(12): 
778–780.  
26.  
Lewis VA, Larson BK, McClurg AB, Boswell RG, Fisher ES. The Promise And Peril Of 
Accountable Care For Vulnerable Populations: A Framework For Overcoming Obstacles. Health 
Aff (Millwood). 2012 Aug 1;31(8):1777–1785. PMID: 22869656 
27.  
Pollack C, Armstrong K. Accountable care organizations and health care disparities. 
JAMA. 2011 Apr 27; 305(16):1706–1707.  
28.  
National Public Radio. Kids With Costly Medical Issues Get Help, But Not Enough. [Last 
accessed 2014 Jan 4]. Available from: 
http://www.npr.org/blogs/health/2013/08/26/214852775/kids-with-costly-medical-issues-get-
help-but-not-enough 
29.  
Ohio Office of Health Transformation. [Last accessed 2014 Jan 4].  Available from: 
http://www.healthtransformation.ohio.gov/LinkClick.aspx?fileticket=LsA-
ATBAmBg%3D&tabid=102 
30.  
Damore JF, DeVore S, Champion RW, Kroch EA, Lloyd DA, Shannon DW. The Many 
Journeys to Accountable Care: 4 Case Studies. The Commonwealth Fund; 2013 Aug. [Last 
accessed 2014 Jan 4]. Available from: 
https://www.premierinc.com/wps/wcm/connect/2d3968e6-b532-491c-8ccf-
2beb6ebfc7b8/Accountable+Care+White+Paper.pdf?MOD=AJPERES 

29 
 
31.  
A Profile of Medicaid Managed Care Programs in 2010: Findings from a 50-State Survey. 
[Last accessed 2014 Jan 4]. Available from: http://kff.org/medicaid/report/a-profile-of-
medicaid-managed-care-programs-in-2010-findings-from-a-50-state-survey/ 
32.  
Social Security Administration. Child Disability Starter Kit Fact Sheet. [Last accessed 2014 
Jan 4]. Available from: 
http://www.ssa.gov/disability/disability_starter_kits_child_factsheet.htm 
33.  
Social Security Administration. Medicaid and the SSI Program. [Last accessed 2014 Jan 
4]. Available from: https://secure.ssa.gov/poms.nsf/lnx/0501715010 
34.  
Health Policy Institute of Ohio. Ohio Medicaid Basics 2013. 2013 March. [Last accessed 
2014 Jan 4]. Available from: http://www.healthpolicyohio.org/ 
35.  
Creswell JW, Clark V. Designing and Conducting Mixed Methods Research. Thousand 
Oaks, CA: Sage; 2007.  
36.  
Patton MQ. Qualitative Research & Evaluation Methods. Thousand Oaks, CA: SAGE; 
2002.  
37.  
Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Sourcebook. 
Thousand Oaks, CA: SAGE; 1994.  
38.  
Saldana J. The Coding Manual for Qualitative Researchers. Thousand Oaks, CA: SAGE; 
2012.  
39.  
Glaser BG. The Constant Comparative Method of Qualitative Analysis. Soc Probl. 1965 
Apr; 12(4): 436–445.  
40.  
Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative 
Research. Aldine Publishing Company; 1967.  
41.  
Song PH, Reiter KL, Weiner BJ, Minasian L, McAlearney AS. The Business Case for 
Provider Participation in Clinical Trials Research: An application to the National Cancer 
Institute’s community clinical oncology program. Health Care Manage Rev. 2012 Oct 5;  
42.  
Song PH, Robbins J, Garman AN, McAlearney AS. High-performance work systems in 
health care, part 3: the role of the business case. Health Care Manage Rev. 2012 Apr 
20;37(2):110–121.  
43.  
Hilligoss B. Selling patients and other metaphors: A discourse analysis of the interpretive 
frames that shape emergency department admission handoffs. Soc Sci Med. 2014 Feb;102:119–
128.  
44.  
Hilligoss B, Zheng K. Chart biopsy: an emerging medical practice enabled by electronic 
health records and its impacts on emergency department–inpatient admission handoffs. J Am 
Med Inform Assoc. 2013 Mar 1;20(2):260–267. PMID: 22962194 
45.  
Agency for Healthcare Research & Quality. CHIPRA Measures by CHIPRA Categories 
[Internet]. [cited 2014 Jan 4]. Available from: 
http://www.ahrq.gov/policymakers/chipra/pqmpmeasures.html 
46.  
National Committee for Quality Assurance. HEDIS 2013 Technical Specifications for ACO 
Measurement [Internet]. 2013. Available from: 
http://www.ncqa.org/Portals/0/HEDISQM/HEDIS2013/ACO_Core_Measure_List_9.6.12.pdf 
47.  
Menachemi N, Blackburn J, Becker DJ, Morrisey MA, Sen B, Caldwell C. Measuring 
Prevention More Broadly: An Empirical Assessment of CHIPRA Core Measures. Medicare 
Medicaid Res Rev. 2013;3(3):E1–E16.  

30 
 
48.  
Kronick R, Gilmer T, Dreyfus T, Lee L. Improving Health-Based Payment for Medicaid 
Beneficiaries: CDPS. Health Care Financ Rev. 2000 Spring;21(3):29–64.  
49.  
PCORI (Patient-Centered Outcomes Research Institute) Methodology Committee. The 
PCORI Methodology Report [Internet]. 2013. Available from: 
http://www.pcori.org/assets/2013/11/PCORI-Methodology-Report.pdf 
 
 

31 
 
Attachments 
 
 
 

32 
 
ATTACHMENT A 
PCORI: Improving Care Coordination for Children with Disabilities Through an 
Accountable Care Organization 
Principal Investigator: Paula H. Song, Ph.D. 
The University of North Carolina at Chapel Hill 
 
Patient Advisory Panel Members 
Updated: January 5, 2015 
The Patient Advisory Panel consists of parent/caregivers of ABD children and/or individuals who 
have professional expertise as advocates for ABD children. The Patient Advisory Panel will be 
led by three study investigators: Sandy Oxley, Tom Scheid and Sandra Tanenbaum.   
 
Member Name  
Qualifications 
Sandy Oxley 
Study Investigator 
CEO, Voices for Ohio’s Children 
Thomas Scheid  
 
Sandra Tanenbaum, PhD 
Study Investigator 
Health Policy Consultant, Voices for Ohio’s Children 
Study Investigator  
The Ohio State University 
 
Maria Beckstedt 
 
Parent/Caregiver  
Geoffrey Collver 
Policy and Communications Director, Disability Rights Ohio 
 
Patty Dovell 
Project Coordinator, Family Voices for Ohio 
 
Becky Fusco 
Parent/Caregiver  
 
Earnestine Hargett 
Senior Disability Rights Advocate, Disability Rights Ohio 
 
Stacy Isenbarg 
Parent/Caregiver  
 
Angi Lee* 
 
Peggy Martin 
 
Government Relations for Easter Seals of Ohio 
 
Parent/Caregiver 
Marla Root 
Parent/Caregiver of ABD Child 
Public Policy Chair & Board Member, Autism Society of Ohio 
 
Candace Knight 
Director of Programs and Services, Easter Seals Central and 
Southeast Ohio 
 
 
*Ms. Lee will serve as an observer of the Patient Advisory Panel.  

1 
 
ATTACHMENT B 
 
Detailed Measure Specifications and Scoring for FECC Caregiver Survey Indicators 
All are on 0-100 scale, where higher is better. Survey response items should be coded to reflect that. For dichotomous items, "no" = 0 and "yes" = 
100. More specific instructions are included where applicable in Scoring Notes. 
All screener items must be nonmissing for a dependent item to be scored 
If items are part of a yes/no checklist, if at least one item is answered, impute "no" for skipped items in mailed surveys or "don't know" for 
telephone surveys (but not for items that telephone respondents refused to answer). 
Otherwise, all component items must be nonmissing for a multi-item indicator to be scored. 
MP=Main Provider 
Indicator ID 
Indicator Description 
Eligible 
Survey Item 
Scoring notes 
CARE COORDINATION SERVICES 
CC2 
Caregivers should report that their 
child has a designated care 
coordinator. 
Q2=1 (Child visited more than one 
doctor’s office or used more than 
one kind of health care service in 
last 12 months) 
Q3 (Someone in MP’s office helped manage 
child’s care or treatment from different 
doctors/providers) 
CC3 
Caregivers who report that their child 
has a designated care coordinator (as 
identified in CC2) should report that 
they know how to access their care 
coordinator. 
Q2=1 (Child visited more than one 
doctor’s office or used more than 
one kind of health care service in 
last 12 months), Q3=1 (Someone 
in MP’s office helped manage 
child’s care or treatment from 
different doctors/providers) 
Q6 (Knew how to contact person who helped 
manage child’s care when you needed help or 
had a question) 
CC5  
Caregivers who report having a 
designated care coordinator (as 
identified in CC2) and who require 
community services should also 
report that their care coordinator 
helped their child to obtain needed 
community services in the last year.  
Q2=1 (Child visited more than one 
doctor’s office or used more than 
one kind of health care service in 
last 12 months), Q3=1 (Someone 
in MP’s office helped manage 
child’s care or treatment from 
different doctors/providers), 
Q15=1 (Caregiver or child needed 
or used community services in last 
12 mos)  
 
Q16(Person in MP’s office who helped manage 
child’s care helped get community services)  

2 
 
CC7  
Caregivers who report having a care 
coordinator (as identified in CC2) 
should also report that their care 
coordinator has contacted them (via 
face-to-face contact, telephone, 
email, or written correspondence) or 
attempted to contact them at least 
once in the last 3 months.  
Q2=1 (Child visited more than one 
doctor’s office or used more than 
one kind of health care service in 
last 12 months), Q3=1 (Someone 
in MP’s office helped manage 
child’s care or treatment from 
different doctors/providers)  
Q7 (In last 3 mos, person contacted you w/o 
you getting in touch w/them first)  
CC8  
Caregivers of children with complex 
needs who report having a care 
coordinator and who report that 
their care coordinator has contacted 
them in the last 3 months should also 
report that their care coordinator 
asked them about the following:  
Q2=1 (Child visited more than one 
doctor’s office or used more than 
one kind of health care service in 
last 12 months), Q3=1 (Someone 
in MP’s office helped manage 
child’s care or treatment from 
different doctors/providers), Q7=1 
(In last 3 mos, care coordinator 
contacted you w/o you getting in 
touch w/them first)  
Mean of CC8a and CC8b 
CC8a 
Caregiver concerns 
Q9 (Care coordinator asked about caregiver concerns)  
CC8b 
Health changes of the child 
Q10 (Care coordinator asked about health changes of child) 
CC8c 
Caregivers who report the 
following: having a designated 
care coordinator, having a copy 
of a written shared care plan for 
their child, and having been 
contacted by their care 
coordinator in the last 3 months 
should also report that their 
care coordinator asked them 
about the following: Progress 
towards goals documented in 
the patient's shared care plan 
Q2=1 (Child visited more 
than one doctor’s office 
or used more than one 
kind of health care 
service in last 12 
months), Q3=1 (Someone 
in MP’s office helped 
manage child’s care or 
treatment from different 
doctors/providers), Q7=1 
(In last 3 mos, care 
coordinator contacted 
you w/o you getting in 
Q31 (In last 12 mos, has 
MP or anyone from MP’s 
office talked w/you about 
progress child was 
making toward goals 
written in shared care 
plan) 
If Q31=3 (Child’s shared 
care plan does not have 
written goals), score as 
No 

3 
 
touch w/them first), 
Q29=1 (MP created 
shared care plan for 
child), Q30=1 (Caregiver 
has copy of child’s shared 
care plan) 
CC9 
Caregivers who report their 
child was referred to see a 
specialist in the last 12 months 
and who report having a care 
coordinator for their child 
should also report that the care 
coordinator contacted them to 
confirm they were able to get 
an appointment with the 
specialist 
Q2=1 (Child visited more 
than one doctor’s office 
or used more than one 
kind of health care 
service in last 12 
months), Q3=1 (Someone 
in MP’s office helped 
manage child’s care or 
treatment from different 
doctors/providers), 
Q13=1 (MP told you child 
needed to see specialist 
during last 12 mos) 
Q14 (Person in MP’s 
office who helped 
manage child’s care 
contacted you to make 
sure child got 
appointment to see 
specialist) 
If Q14=3 (Did not get 
help managing child’s 
care), score as No 
 
CC10 
Caregivers who report 
having a care coordinator 
should also report that 
their care coordinator: 
Q2=1 (Child visited more 
than one doctor’s office 
or used more than one 
kind of health care 
service in last 12 
months), Q3=1 (Someone 
in MP’s office helped 
manage child’s care or 
treatment from different 
doctors/providers) 
Q5 (In last 12 mos, 
person in MP’s office 
who helped manage 
child’s care…)  
Give partial credit (No=0, 
Yes Somewhat=50, Yes 
Definitely=100), take 
mean of CC10a-c  
CC10a 
Was knowledgeable about their 
Q5a, Q5b 
Mean of rescaled Q5a and Q5b  

4 
 
child’s health 
CC10b 
Supported the caregiver 
Q5c  
CC10c 
Advocated for the needs of the 
child  
Q5d, Q5e 
Mean of rescaled Q5d and Q5e 
MH8  
Caregivers or patients 
who self-identify as 
having a preference for 
conducting medical visits 
in a language other than 
English should have 
access to a professional 
medical interpreter (live 
or telephonic) at all visits 
for which an interpreter 
is needed. 
Q39=1 (Speak language 
other than English at 
home), Q41=2 (Prefer to 
talk w/child’s doctors in 
language other than 
English), Q44=2, 3, or 4 
(Needed professional 
interpreter during visit to 
MP some, most, or all 
visits) 
Q45 (How often was 
professional interpreter 
available when needed) 
Partial credit: rescale Q45 
to 0-100Q45=1 (Never): 0 
points, Q45=2 
(Sometimes): 100/3 
points, Q45=3 (Usually): 
100*2/3 points,Q45=4 
(Always): 100 points 
MESSAGING  
IE2 
Caregivers/patients who 
report receiving a written 
visit summary during the 
last 12 months from their 
child’s MP’s office should 
report that it contained 
the following elements: 
Q17=1 (received written 
visit summary in last 12 
mos) 
Q18 (How often did 
written visit summaries 
include…) 
Give partial credit 
(Never=0, Sometimes=50, 
Always=100), take mean 
of IE2a-f  
IE2a 
Current problem list 
Q18a  
IE2b 
Current medication list 
Q18b (Rx), Q18c (OTC) 
Mean of rescaled Q18b and 
Q18c 
 
IE2c 
Drug allergies 
Q18d (list of “child’s allergies”, not drug 
allergies specifically)  
IE2d 
Specialists involved in the child's care 
Q18e  
IE2e 
Planned follow-up 
Q18f  
IE2f 
What to do for problems related to the 
Q18g 

5 
 
outpatient visit 
 
IE3 
Caregivers/patients who reported 
ever receiving a visit summary in 
the last 12 months from their 
child’s MP’s office (as identified in 
IE2) should report that the 
summary: 
Q17=1 (received written visit 
summary in last 12 mos) 
Give partial credit (Never=0, 
Sometimes=50, Always=100), take 
mean of IE3a and IE3c  
IE3a 
Was easy to understand 
Q19  
IE3c 
Was useful 
Q20  
IE4 
Caregivers should report having 
been invited to join in hospital 
rounds during their child’s last 
hospitalization 
 
Q21=1 (child had overnight 
hospital stay in last 12 mos) 
Q25 (Last time child was in 
hospital, invited to take part in 
hospital rounds) 
IE5 
Caregivers/patients should 
report receiving a written 
visit summary of their 
child’s last hospitalization 
at the time of discharge, 
and they should report the 
summary contained the 
following elements: 
 
Q21=1 (child had overnight 
hospital stay in last 12 
mos), Q22=1 (Last time 
child was in hospital, got 
written hospital stay 
summary at discharge) 
Q23 (Did written hospital 
stay summary include…) 
Mean of IE5a-f  
IE5a 
Problem list at time of discharge 
Q23a  
IE5b 
Medication list at time of discharge 
Q23b (Rx), Q23c (OTC) 
Mean of rescaled Q23b and Q23c 
 
IE2c 
Drug allergies 
Q18d (list of “child’s allergies”, not drug allergies specifically)  
IE2d 
Specialists involved in the child's care 
Q18e  
IE2e 
Planned follow-up 
Q18f  
IE2f 
What to do for problems related to the 
outpatient visit 
 
Q18g 

6 
 
IE3 
Caregivers/patients who reported ever 
receiving a visit summary in the last 12 
months from their child’s MP’s office (as 
identified in IE2) should report that the 
summary: 
Q17=1 (received written visit 
summary in last 12 mos) 
Give partial credit (Never=0, 
Sometimes=50, Always=100), take 
mean of IE3a and IE3c  
IE3a 
Was easy to understand 
Q19  
IE3c 
Was useful 
 
Q20  
IE4 
Caregivers should report having been 
invited to join in hospital rounds during 
their child’s last hospitalization 
 
Q21=1 (child had overnight 
hospital stay in last 12 mos) 
Q25 (Last time child was in 
hospital, invited to take part in 
hospital rounds) 
IE5 
Caregivers/patients should 
report receiving a written 
visit summary of their 
child’s last hospitalization 
at the time of discharge, 
and they should report the 
summary contained the 
following elements: 
 
Q21=1 (child had overnight 
hospital stay in last 12 
mos), Q22=1 (Last time 
child was in hospital, got 
written hospital stay 
summary at discharge) 
Q23 (Did written hospital 
stay summary include…) 
Mean of IE5a-f  
IE5a 
Problem list at time of discharge 
Q23a  
IE5b 
Medication list at time of discharge 
Q23b (Rx), Q23c (OTC) 
Mean of rescaled Q23b and Q23c 
 
IE5c 
Drug allergies 
Q23d (list of “child’s allergies”, not drug 
allergies specifically)  
IE5d 
Specialists involved in the child's care during the 
hospitalization 
Q23e  
IE5e 
Planned follow-up 
Q23f  
IE5f  
What to do for problems related to the hospitalization 
Q23g 
IE6 
Caregivers who received a written 
summary of their child’s 
hospitalization at discharge (as 
Q21=1 (child had overnight 
hospital stay in last 12 mos), 
Q22=1 (Last time child was in 
Q24 (Hospital stay summary 
easy to understand) 

7 
 
described in indicator IE5) should 
report that the information contained 
in the visit summary was easy to 
understand 
 
hospital, got written hospital 
stay summary at discharge) 
MH3a 
Caregivers should report 
having access to an 
electronic health record 
to look up information 
about their child’s visits 
and health care. 
 
All caregivers of children 
with medical complexity 
Q26 (In last 12 mos, MP’s 
office had web site or 
app caregiver could use 
between visits to look up 
information about child’s 
visits and care) 
If Q26=1 (Yes): 100 
pointsIf Q26=2 or 3 (No 
or Don’t Know): 0 points  
MH3b 
Caregivers who report 
having access to an 
electronic health record 
should also report that it 
includes the following 
health information: 
Q26=1 (In last 12 mos, MP’s office had web 
site or app caregiver could use between visits 
to look up information about child’s visits and 
care) 
Do not score the N/A's; If no 
immunizations (meds), do not 
score MH3b1 (MH3b2); If didn't 
look at med list, do not score 
MH3b2Take mean of MH3b1 
and MH3b2 
 
MH3b1 
Immunization record 
Q26=1 (In last 12 mos, MP’s 
office had web site or app 
caregiver could use 
between visits to look up 
information about child’s 
visits and care) 
Q27 (In last 12 mos, MP’s 
web site or app had list of 
immunizations child has 
received)  
If Q27=3 (no immunizations 
in last 12 mos): do not 
scoreIf Q27=1 (Yes): 100 
pointsIf Q27=2 or 4 (No or 
Don’t Know): 0 points  
MH3b2 
List of child’s medications 
Q26=1 (In last 12 mos, MP’s 
office had web site or app 
caregiver could use 
between visits to look up 
information about child’s 
visits and care) 
Q28 (In last 12 mos, MP’s 
web site or app had list of 
child’s meds) 
If Q28=3 (no meds in last 12 
mos): do not score MH3b2If 
Q28=1 (Yes): 100 points for 
MH3b2If Q28=2 or 4 (No or 
Don’t Know): 0 points for 
MH3b2 

8 
 
MH6 
Caregivers who report their 
child’s condition causes 
difficulty learning, 
understanding, or paying 
attention in class should 
also report that one of their 
child’s health care 
providers (i.e., primary care 
physician, specialist 
physician, care coordinator, 
NP, nurse, social worker, 
etc) communicated with 
school staff at least once a 
year about the educational 
impacts of the child’s 
condition. 
Q35=1 (Child attended 
school in last 12 mos), 
Q36=1 (Because of health 
condition child has 
difficulty learning, 
understanding, or paying 
attention in class) 
Q37 (In last 12 mos, 
someone from MP’s office 
contacted staff at child’s 
school to make sure they 
understood how child’s 
condition affected ability to 
learn, understand, or pay 
attention) 
If Q37=1 (Yes): 100 pointsIf 
Q37=2 or 3 (No or Don’t 
Know): 0 points 
PROTOCOLS/PLANS 
SCP2 
Caregivers should report 
that their child’s primary 
care provider created a 
shared care plan for their 
child. 
All caregivers of children 
with medical complexity 
Q29 (MP created shared 
care plan for child) 
Q29=1 (Yes): 100 points 
Q29=2 (No): 0 points 
 
SCP4 
Caregivers of children (age 
15 years or older) should 
report that their child’s 
main provider created a 
written transition plan for 
their child. 
Q33=1 (Child age 15 or 
older) 
Q34 (MP created transition 
plan for child) 
Q34=1 (Yes): 100 
pointsQ34=2 or 3 (No or 
Don’t Know): 0 points 
SCP6 
Caregivers should report 
that their child’s main 
provider created an 
emergency care plan for 
their child.  
All caregivers of children 
with medical complexity 
Q32 (MP created 
emergency care plan for 
child) 
Q32=1 (Yes): 100 
pointsQ32=2 (No): 0 points 

9