A Randomized, Controlled Trial of a Telephone-Based Developmental Care Coordination System
Phase: NA
Conditions: Child Development, Health Services Research
Sponsor: University of California, Los Angeles

Study Protocol with SAP 
 
 
A Randomized, Controlled Trial of a Telephone-Based Developmental Care Coordination 
System 
 
NCT04118452 
PI: Paul J. Chung MD MS 
Document Date: 3/28/2023 
 
 
 

Abstract 
Background 
The American Academy of Pediatrics (AAP) recommends routine, universal developmental 
surveillance at all well-child visits for children from birth to 5 years old; universal screening with 
a structured, validated tool at the 9-, 18-, and 30-month visits; and Autism-specific screening at 
the 18- and 24-month visits. These recommendations are based on decades of research that 
demonstrate long-term benefits of high-quality early interventions. Despite these 
recommendations, only about 30% of pediatricians nationally conduct universal developmental 
screening, and children who screen positive often never receive formal evaluation or treatment 
until after kindergarten entry, when interventions may have less impact on long-term outcomes.  
Multiple barriers prevent children with developmental risks from receiving services that might 
ultimately help them achieve optimal development. Beyond initial screening, children must be 
appropriately referred to service organizations, be evaluated by these organizations, be deemed 
eligible for services, and access these services. At every step along this series there is the 
potential for leakages, or “voltage drops.” Preliminary findings suggests that 2-1-1 Los Angeles 
County (211LA), part of the national 2-1-1 health and human services call center network, can 
address these barriers and reduce these voltage drops, connecting more children to services 
and potentially improving outcomes. 
Objectives 
This study examines: 1) the effectiveness of telephone-based early childhood developmental 
care coordination through 211LA in terms of rates of a) referral to evaluation by early 
intervention services, b) eligibility for early intervention services, and c) receipt of services, 
compared to usual care; 2) medium-term child developmental outcomes in the 211LA group, as 
determined by formal developmental assessment 2 years after enrollment, compared to usual 
care; and 3) the costs of 211-based care coordination, and also the potential long-term benefits 
in terms of improved educational, employment, and other outcomes. 
Methods 
The study design is a randomized controlled trial (RCT) of development-related care 
coordination through 211LA, compared to usual care, in community clinic populations. All 
intervention and control children will receive validated screening, which allows examination to 
focus comparisons on post-screening care coordination through 211LA vs. usual care. 
We will aim to recruit 685 children ages 11-42 months—approximately 150-200 from each of 
four clinic systems – over approximately 16 months. The children will be individually randomized 
to usual care + 211LA + developmental screening results and usual care + developmental 
screening results. Based on our pilot trial, we expect to lose roughly 20% per year.. 
Participants  
Potential subjects are identified through review of the Electronic Health Record in a two-stage 
process. 
First, all clinic patients within the study age range (11-42 months), or who will be in study age 
range within 3 months, will be sent an Opt Out letter by each respective clinic. This process will 
be repeated every 3 months to identify new patients who have joined the clinic or aged into the 
study. This approach is being utilized to minimize time burden on the clinics as well as to allow 
the study to potentially include last minute / short notice appointments. This helps ensure that 

families that do not schedule well child checks far in advance may still be able to be included in 
the study – avoiding systematic bias. 
The Opt Out letter that will be sent by each clinic briefly describes the study and provides the 
study team's contact information so that potential participants can call to ask questions or opt 
out. This letter will also be available at the front desk at clinic sites to increase the reach of this 
highly mobile population 
Second, the clinics will provide the Research team weekly to monthly with an appointment list of 
all potentially eligible children, including: child's name, age (DOB), gender, preferred language, 
assigned primary care provider, parent’s name, and phone number as well as the date of their 
upcoming well child check. 
Procedures 
Recruitment, Screening, Consent, and HIPAA Authorization 
Upon receipt of the patient lists, the names will be entered into REDCap and the study team will 
call through the list in order until the quota for each of the four clinic systems, spread across 10 
sites throughout Los Angeles County, is reached. When called, the RA will provide information 
about the study to each family / parent as well as provide them with an opportunity to ask 
questions. In addition, study flyers will be posted in each of the clinic system's waiting rooms. 
RAs will attempt to contact families up to 10 times - either by phone or by text during recruitment 
- unless additional contact is specifically requested by the family. 
If a family is interested and elects to be screened for study eligibility, the RA will conduct the 
eligibility screening over the phone. The following eligibility criteria will be assessed during the 
screening process.  
1) a parent or legal guardian who is at least 18 years of age, 
2) a parent or legal guardian comfortable participating in the study in English or Spanish, 
3) a child 11-42 months of age with an upcoming well child check, 
4) a child scheduled for a 1- to 3-year old well child check-up (appointments are often 
scheduled early or late in comparison to child's actual age) 
5) a child who is not receiving or has not received developmental or behavioral services, 
6) a child who has not received a developmental or behavioral diagnosis, and 
7) a child who has not been referred for developmental or behavioral services or 
evaluation. 
8) Run-in phase (UCLA IRB #18-000465) study participants will also be excluded - this 
will be determined before a potential participant is called. 
9) Children with a sibling currently or previously participating in this phase of the study 
will also be excluded - this will be reviewed by study record prior to calling the potential 
participant and will also be confirmed by the screening script. 
If eligible, the RA will review the study with the family over the phone and obtain oral consent as 
well as offer to text or email the study information sheet. In addition, the RA will send via email 
or text (based on the family/respondent preference) a link to the UCLA External REDCap 
platform for each family to complete and sign electronically a HIPAA authorization form for 
research for medical record access to allow for later record review and abstraction. HIPAA 
authorizations forms completed electronically through the External REDCap platform, will be 
downloaded and provided to each clinic via secure fax, secure email, or printed in hard-copy - 
based on each clinic system's preferences. The process for obtaining HIPAA research 

authorization at the participating clinics as outlined is acceptable and compliant with the 
participating clinic's HIPAA policies and procedures. 
Baseline Data Collection – Developmental Screening 
The RA will administer the age-appropriate screening survey over the phone using the peds-
test-online, which includes the Modified Checklist for Autism in Toddlers, Revised (M-CHAT-
R™) for children at least 16 months of age, the Parents' Evaluation of Developmental Status 
(PEDS), and the Parents' Evaluation of Developmental Status: Developmental Milestones 
(PEDS:DM). The peds-test online system scores parental responses to each of the 
administered questions and provided screening results for each of the individual instruments. 
Screening results will be provided verbally to each parent. An overall risk level will be calculated 
using all 3 measures – high risk will be defined as PEDS Path A or an M-CHAT-R failure, 
moderate risk will defined as PEDS Path B or 3 unmet milestone on the PEDS:DM, low risk will 
include Path C and E with a passed M-CHAT-R and with 2 unmet milestones or fewer on the 
PEDS:DM. 
Randomization: 
Randomization will be stratified by developmental screenings result (high/moderate vs. low) as 
well as by clinic system, creating 8 groups that will be divided between the intervention and 
control groups to ensure even representation. To determine group assignment, the Research 
Assistant will enter the developmental screening result and clinic into REDCap, which will 
calculate the risk level as well as assign group. The Research Assistant will provide the 
participant with their group assignment at this time as well as describe briefly the intervention 
further for the intervention group. In addition, the Research Assistant will remind the participant 
that a summary of the results will be provided to their doctor. 
Baseline Developmental Assessments: 
The Research Assistant will then administer two developmental assessments over the phone: 
the Child Behavior Checklist (CBCL) and the PEDS:DM-Assessment Level (AL). The 
subdomains of the CBCL the RA will administer include: attention problems, aggressive 
behavior, autism spectrum problems, and oppositional defiant problems. All children will be 
asked all CBCL questions. The PEDS:DM-AL includes several subscales: fine motor, self-help, 
receptive language, expressive language, gross motor, social emotional, and academic. 
Together, they combine to form the cognitive overall score, which is used to calculate age 
equivalency. All low risk kids will receive the expressive and receptive language scale 
questions. Moderate and high risk kids will receive the full PEDS:DM-AL. 
In addition, the RA will ask about families’ experiences with care at the clinic and anticipatory 
guidance received; sociodemographic questions, including covariates such as child gender, 
parent education level, parent marital status, race/ethnicity, language, household composition 
and income, health insurance status, as well as a hunger scale and the PHQ2; a financial 
wellbeing scale, and the protective factors survey. 
Engagement: 
To keep families engaged and remind them of study participation over the 24-month period, 
parenting tips and child health information will be provided by text monthly – texts will be 
relevant to the full age range of the kids enrolled at any time point (11 months to 5 years) and 
will include topics such as sleep, screen time, safety, and nutrition. In addition, a reminder letter 
asking for updated contact information will be sent 1-2 months prior to each survey time point. A 

text reminder will also be sent approximately 2 weeks before each follow-up data collection time 
point. 
Study Conditions 
Intervention: 
Those in the intervention group will then be connected to 211LA via warm transfer phone call for 
care coordination, including a review of developmental screening responses and information 
about and connection to relevant developmental and behavioral evaluations and services. The 
care coordinator specialist will stay in touch with the family for 3-6 months or until the parent 
wishes to discontinue contact or the child is enrolled in services. The RA will also text a 211LA 
flyer and contact information to the parent. The RA will provide participant contact information 
(child and parent name, phone number, address) and basic demographic information (child age, 
race/ethnicity, language preference, education, income, insurance status, household size) to the 
care coordination specialist via an online encrypted WebForm hosted on 211LA's HIPAA 
compliant care coordination platform / database. 
After connection to 211LA, the care coordination specialist will develop a care coordination plan 
– including referrals suggested for community agencies – and will fax this plan to the clinic. At 
the conclusion of care coordination, the care coordination specialist will provide the outcome of 
each referral, including services received and barriers faced. This data will also be transferred to 
the research team via encrypted electronic transfer on an ongoing basis – along with care 
coordination time investment per participant. 
After baseline data collection, parents will receive a $25 cash incentive to compensate them for 
their time. This incentive will be mailed to study participants along with a copy of their HIPAA 
authorization and a copy of the study information sheet. 
Measures 
Follow-up Data Collection: 
Participants will be contacted 3 more times after baseline for study research data collection – 6 
months, 12 months, and 24 months after baseline. Follow-up data collection will be very similar 
to baseline data collection and parents will receive $15, $25, and $25 cash incentives to thank 
them for their time. For hard-to-reach participants, we will add a small additional incentive of $10 
dollars in value at any time point (6M, 12M, 24M). 
The 6-month survey includes the M-CHAT-R for children who were too young at baseline to 
receive the screening. No other developmental screenings will be administered at 6, 12, or 24 
months. The Child Behavior Check List will be administered at 6, 12, and 24 months and the 
PEDS:DM-AL will be administered in the same fashion as baseline at 12 and 24 months, as will 
experiences with care and anticipatory guidance. At all follow-up time points, participants will be 
asked about referrals and connections to services and those in the intervention group will be 
asked about their experiences with 211 at 6 months. At each follow-up time point, the RA will 
call and text a study participant up to 20 times each (10 calls, 10 texts) to connect for the 
interview, unless otherwise specified by the family. We will contact by email study participants 
that we have already contacted by phone but haven't been able to reach to complete the follow-
up surveys. 
Chart Abstraction: 

Medical record review will occur at baseline and throughout the study time period. The RA will 
conduct electronic health record reviews for each of the study participants They will assess 
developmental surveillance, developmental screening (validated screening tool use), and 
documentation of referrals and services from birth to baseline as well as in the 24 months of the 
study period. After completing site-specific training and documentation, the RA will be provided 
with an administrative login for each site's electronic health / medical record. They will use this 
to log in to access the electronic medical record and review the study participant's record for the 
noted elements. In addition, the RA will review for process measures – including the attendance 
of the scheduled baseline well child check-up, upload of developmental screening results into 
the medical record, as well as upload of the care plan and care coordination outcome summary. 
Statistical and Data Analysis Plan:  
Given that the intervention being tested requires parent adherence to recommendations (e.g., 
communicating with 211LA, keeping referral and service appointments) to achieve success, we 
will use an intention-to-treat analysis. Our analyses will be based on measuring differences 
between the intervention and comparison groups with respect to the above outcome variables. 
Additionally, covariate-adjusted differences will be evaluated using multivariable regression 
models. Incomplete variables will be handled using a multiple imputation approach with 10 
imputations for multivariable analyses; imputations will be based on an iterative Markov chain 
Monte Carlo method, with initial values generated by an expectation–maximization algorithm. 
Finally, secondary analyses for all outcomes will include group*developmental screening risk 
category (low, moderate, high), or group*gender (female, male) interactions to explore 
differential intervention effects. 
Hypothesis 1: A higher proportion of families in the intervention group will a) be successfully 
referred to early childhood service organizations for evaluation, and b) receive services, than 
families in the usual care group. 
We will use multivariable logistic regression models for each of these binary outcome variables 
with group assignment as the main predictor, controlling for socio-demographic variables. 
Secondary analyses will use the fact that multiple types of services may be indicated. This will 
allow us to examine specific services in isolation as binary outcome variables in logistic 
regressions. It is possible that certain types of services are more likely to be recommended or 
taken up than others. It is also possible that receipt of any vs. no services is too crude to 
distinguish among different types or severities of developmental needs. Thus, examining 
specific services may prove to be important adjunct analyses. 
Hypothesis 2: Over time, formal developmental assessment scores in the intervention group will 
improve more than scores in the usual care group. 
In all children, linear regressions will be used to compare PEDS:DM-AL language subscale 
scores in each arm. In the subset of children who screened positive and therefore received the 
full PEDS:DM-AL (Parents' Evaluation of Developmental Status: Developmental Milestones - 
Assessment Level), linear regressions will be used to compare each of the subscale scores in 
each arm. Since we will have baseline and 2-year follow-up PEDS:DM-AL scores, we will use 
change from baseline scores as outcomes, allowing us to compare change in the intervention 
group against change in the comparison group. 
We will also conduct exploratory secondary analyses, each with probative value but less certain 
power. Since we will have scores measured at baseline and 1-year and 2-year follow-ups, we 
will be able to conduct longitudinal analyses, with multiple observations clustered within 
individuals using linear mixed models with random child effects. These models will adjust for 

socio-demographic variables. The outcome variables will be appropriately transformed prior to 
fitting the model as needed. 
The mixed effects models described above allow us to make statistical inference using all 
observed data, with the assumption of data missing at random. To assess the robustness of this 
analysis with respect to missing data, multiple imputation with pattern mixture models will also 
be used to impute missing data due to missed visits or loss to follow-up. For example, as a 
sensitivity analysis, we will assume that the distribution of missing data from missed visits or 
loss to follow-up is similar with that of the usual care group. 
Secondary outcomes will include CBCL, family-centered care, and satisfaction scores. We will 
conduct longitudinal analyses like those for the language subscales. Also, sensitivity analyses 
clustering children by provider will be performed to assess robustness of findings to potential 
contamination by provider. 
Hypothesis 3: Intervention costs per child screened will be exceeded by the societal value of the 
long-term benefits from 211-based care coordination. 
We will compare costs per child screened to benefits. Total costs will be disaggregated into RA 
screening/initial referral, 211 program, and diagnostic and developmental services costs. Total 
benefits will be disaggregated into income and value of life expectancy gains. Using evidence 
from literature, these benefits will be estimated by predicting changes in educational 
achievement from improvements in early life assessments (Aim 2 findings), then predicting 
changes in educational attainment from changes in educational achievement, and then 
predicting changes in lifetime income and life expectancy from changes in educational 
attainment. Sensitivity analyses will explore alternative assumptions about costs, relationships 
between early-childhood development and long-term outcomes, and value of a life-year. 
Power Analysis 
Given our planned RCT enrollment of 662 families (331 per group), and assuming loss to follow-
up of 20% per year, we expect 424 families (212 per group) to complete the study. For each 
hypothesis, we report minimally detectable differences using study completers. These estimates 
are likely conservative, since some analyses will be performed using data collected prior to the 
end of follow-up and including >424 families. The enrollment sample size was chosen so that 
detectable differences would be within the bounds of the confidence intervals from the pilot 
data. Unless otherwise noted, reference rates were estimated from the pilot study. 
Hypothesis 1: A higher proportion of families in the intervention group will a) be successfully 
referred to early childhood service organizations for evaluation, b) be deemed eligible for 
services by service organizations, and c) receive services, than families in the usual care group. 
a) A sample size of 212 per group provides 80% power to detect a 11.9 percentage-point 
difference in the referral rate, assuming a reference rate of 20.4%, a chi-squared test, and an 
alpha of 0.05. This difference is smaller than observed in the pilot study (21.8 percentage 
points; 95% CI: 9.1-33.8). 
b) A sample size of 212 per group provides 80% power to detect an 9.2 percentage-point 
difference in the eligibility rate, assuming a reference rate of 8.6%, a chi-squared test, and an 
alpha of 0.05. This difference is smaller than observed in the pilot study (14.3 percentage 
points; 95% CI: 5.6-24.0). 
c) A sample size of 212 per group provides 80% power to detect a 8.7 percentage-point 
difference in the service receipt rate, assuming a reference rate of 7.2%, a chi squared test, and 
an alpha of 0.05. This difference is smaller than observed in the pilot study (11.7 percentage 
points; 95% CI: 3.4-21.0). 

Hypothesis 2: Over time, formal developmental assessment scores in the intervention group will 
improve more than scores in the usual care group. 
Meta-analyses of interventions studies have shown a range of effect sizes (Cohen’s d), from 
0.3-0.8 for speech and language therapies, and 0.3-1.19 for Autism Spectrum Disorder. A 
sample size of 212 per group provides 80% power to detect an effect size as small as 0.27 on 
the PEDS:DM-AL, assuming a two-sample t-test, and a two-sided alpha of 0.05.