Adaptive Trial of Parent Empowerment and Coaching in Early Intervention: PEACE for ImPACT Study
Phase: NA
Conditions: Autism Spectrum Disorder
Sponsor: University of Pennsylvania

ADAPTIVE TRIAL OF PARENT 
EMPOWERMENT AND 
COACHING IN EARLY 
INTERVENTION 
PEACE FOR IMPACT STUDY 
Principal Investigator: Melanie Pellecchia, PhD 
Melanie.pellecchia@pennmedicine.upenn.edu 
Research Protocol
Document Date: 11/30/2023 

Project PEACE for ImPACT Research Protocol 
 
Page 1 of 20 
Version 2.0 
 
 Adaptive Trial of Parent Empowerment and Coaching in Early Intervention: 
PEACE for ImPACT Study 
RESEARCH PROTOCOL 
 
BACKGROUND 
Caregiver-mediated interventions are considered an evidence-based practice (EBP) for young 
children with autism. Caregiver coaching is key to efficacious caregiver mediated early 
intervention (EI) for autism. Caregiver-mediated interventions include caregivers as partners in 
delivering EI services through explicit caregiver coaching. Coaching caregivers increases 
parents’ use of intervention strategies, and caregivers overwhelmingly appreciate getting 
coached (Abouzeid et al., 2020; Siller et al., 2022). Families of toddlers who received caregiver 
coaching used fewer services than families receiving usual community-based services, 
suggesting significant cost savings (Tsiplova et al., 2022). Coaching is an interactive process 
between a provider and a caregiver involving observation, reflection, and action to promote the 
caregiver’s ability to support their child’s participation at home and in their community. Despite 
policies supporting caregiver coaching, early intervention providers rarely coach 
caregivers of children with ASD (Campbell & Coletti, 2013; Douglas et al., 2020; Salisbury et 
al., 2012). Instead, they usually work directly with the child (Campbell & Coletti, 2013; 
Campbell & Sawyer, 2007). In contrast to university-based studies of caregiver-mediated 
interventions, community early intervention outcomes tend to be poor, especially in low-income 
communities (Nahmias et al., 2019). Poorer outcomes may be due in part to a lack of effective 
parent coaching. 
 
The Philadelphia EI system launched a system-wide initiative to train EI providers in Project 
ImPACT, an evidence-based caregiver coaching model for families of young children with 
autism (Ingersoll & Dvortcsak, 2019). However, our previous evaluations indicated that 
intervention fidelity was low, meaning most providers did not implement the intervention as 
designed. We partnered with the EI system to evaluate the reasons for low intervention fidelity, 
and we developed an implementation toolkit called Parent Empowerment and Coaching in Early 
Intervention (PEACE) to improve the implementation of caregiver coaching for families of 
young children with autism. We conducted a pilot study of the PEACE toolkit and our pilot test 
indicated that the PEACE toolkit shows promise for improving providers’ use of caregiver 
coaching, increasing parents’ responsive parenting, and improving child outcomes. However, the 
sample was small, and some providers required additional implementation support. The goal of 
this study is to test the impact of PEACE and determine the level of implementation 
support needed to improve provider fidelity to caregiver coaching, which will subsequently 
improve caregiver and child outcomes.  
 
RESEARCH PLAN 
We will use a Hybrid Type 3 implementation-effectiveness trial (Curran et al., 2012) using a 
SMART design (Almirall et al., 2018; Nahum-Shani & Almirall, 2019) to efficiently test the 
effect of increased implementation support for providers who demonstrate sub-threshold levels 
of coaching adherence. A SMART design provides a systematic and principled way to monitor 
and rapidly improve providers’ caregiver coaching adherence. Rapid assessment of coaching 

Project PEACE for ImPACT Research Protocol 
 
Page 2 of 20 
Version 2.0 
adherence and assignment to increasing implementation support will enable the rapid support for 
families whose providers are not meeting the standard for implementation, providing them with 
an immediate step-up in their care. This study will address four aims:  
Aim 1: Identify the level of implementation support needed to rapidly improve providers’ use of 
caregiver coaching with a sample of 200 EI providers. 
Aim 2: Assess caregiver and child outcomes for families receiving caregiver coaching through 
Project ImPACT for a sample of 400 families, 
Aim 3: Identify provider and family characteristics associated with the need for increasing 
implementation support 
Aim 4: Evaluate the cost and cost-effectiveness of tiered implementation strategies.  
Setting  
In Philadelphia, children less than five years of age with or at high risk for developmental 
disabilities are eligible for publicly funded early intervention. Each year, approximately 1,700 
children with ASD receive early intervention through agencies that contract with the County to 
provide these services. Services are delivered in the family’s home or preschool/daycare setting. 
 
Participants 
 
Providers: We will recruit 200 providers from participating EI agencies across four years. All 
providers have a Bachelor’s or Master’s degree in a relevant field (e.g., psychology, education, 
speech pathology). They often are independent contractors. We will recruit providers in years 1-
4 (~50 per year) by randomly sampling without replacement from each agency, to evenly 
distribute providers across agencies each year. Inclusion criteria for providers are: 1) employed 
by a Philadelphia EI agency and 2) have > 5 children with/at risk for ASD on their caseload.  
 
Children and Families: We will recruit 400 children and one caregiver per child (2 parent-child 
dyads per provider). Inclusion criteria are that children must: 1) be receiving EI services from a 
participating provider; 2) have a classification of ASD or high ASD risk as determined by the EI 
system; 3) caregiver speaks English or Spanish (Project ImPACT materials are available in 
Spanish); and 4) have a caregiver willing to participate in weekly coaching sessions.  
 
Methods.  
 
Intervention – Project ImPACT: Project ImPACT is a manualized, caregiver-mediated, 
naturalistic developmental behavior intervention that includes two core components: (1) a child-
directed curriculum to guide caregivers in supporting their child’s social communication; and (2) 
guidelines to help EI providers coach caregivers. Project ImPACT training includes a 6-hour 
self-paced e-course and 14-hr interactive workshop on the child-directed Naturalistic 
Developmental Behavioral Intervention components of the program, the scope and sequence of 
the curriculum, data collection and progress monitoring strategies, and strategies to support 
caregivers in learning the intervention strategies.  
 
PEACE Implementation Toolkit Description: The PEACE Toolkit is a theory-informed, modular 
toolkit of strategies that map onto identified implementation barriers to caregiver coaching. The 

Project PEACE for ImPACT Research Protocol 
 
Page 3 of 20 
Version 2.0 
PEACE toolkit has three parts:  
 
Tier 1: PEACE online resource library. The PEACE online resource includes self-paced training 
modules that cover the foundations of caregiver coaching, caregiver perspectives towards 
coaching, strategies for aligning with caregivers and gaining buy-in for coaching, evidence-based 
strategies to empower caregivers from marginalized backgrounds, and strategies to improve 
collaboration and cohesion among providers working with the same child. Each module includes 
brief example videos, easy-to read infographics, tip sheets, and checklists to guide 
implementation. A virtual communication application, Slack, facilitates frequent communication, 
enhances social networks among providers, and creates social norms related to the use of 
caregiver coaching. Providers can post questions or suggestions, share resources, and receive 
updates about other providers’ accomplishments. Consultants frequently post encouraging 
messages and progress updates to increase motivation and shared norms for coaching. 
 
Tier 2: Weekly group facilitation meetings. PEACE also includes facilitation meetings to 
improve provider self-efficacy and fidelity to caregiver coaching strategies. Facilitation 
comprises interactive problem solving and support, a shared understanding of need for 
improvement, and a supportive interpersonal relationship. Effective facilitation has been linked 
to improved implementation outcomes (Powell et al., 2015). Providers participate in 12 weekly 
group facilitation meetings for one hour virtually with an expert consultant focused on auditing 
providers’ fidelity to caregiver coaching through role-play and performance feedback. Each 
group meeting aligns with content from the PEACE online resource library modules. 
 
Tier 3: Individual Facilitation Meetings. Individual facilitation is delivered virtually for one hour 
each week for eight weeks with a consultant focused on reflection and problem solving any 
identified implementation barriers, as well as practice and performance feedback. 
 
Aim 1: Examine the effectiveness of PEACE at tiered implementation levels.  
 
Research Question: How can we optimize implementation supports for EI providers to 
implement caregiver coaching with fidelity?  
 
Overview 
Our primary aim is to evaluate the effectiveness of tiered levels of implementation support in 
increasing providers’ caregiver coaching adherence using the PEACE implementation toolkit 
components (i.e., PEACE online resource library and chat, PEACE group facilitation, PEACE 
individual facilitation). We will conduct two sets of comparisons, based on randomizations to 
increasing implementation support and on the embedded support sequences, to evaluate the 
optimal implementation support needed to improve providers’ caregiver coaching adherence, our 
primary outcome of interest. We also will collect measures of acceptability, feasibility, and 
appropriateness of the PEACE implementation toolkit as secondary implementation outcomes.  
 
Tier 1: PEACE online resource library: All providers will have access to the PEACE Online 
resources following completion of Project ImPACT training.    
Tier 2: PEACE Group Facilitation: After eight weeks (see Exhibit 1), providers with low 

Project PEACE for ImPACT Research Protocol 
 
Page 4 of 20 
Version 2.0 
coaching adherence will be randomized to continue with online resources or to participate in 
weekly group facilitation.  
Tier 3: PEACE Individual Group Facilitation: After 16 weeks, providers may be randomized to 
receive individual facilitation with an expert coach.  
 
 Design.  
The SMART design uses two evaluations of provider coaching adherence, at eight weeks and 16 
weeks of intervention (Exhibit 1). For the first eight weeks, all providers will have access to the 
PEACE online resource library and virtual chat space and will implement Project ImPACT as 
usual. At eight weeks, we will rate each provider’s caregiver coaching adherence, separately for 
each child, using the PEACE Caregiver Coaching Fidelity Tool. If a provider has a caregiver 
coaching adherence score < 80%, for either child, the provider will be randomized 1:1 to receive 
either 1) PEACE weekly group facilitation to improve fidelity; or 2) to continue without support, 
through the 16-week time point. Providers adherent for all children at eight weeks will continue 
without support through 16-weeks. At the 16-week time point, all providers will be evaluated 
again, and non-adherent providers will be randomized 1:1 to receive either 1) PEACE weekly 
group facilitation, or 2) PEACE individual facilitation through the 24-week time point; again, 
providers who demonstrate coaching adherence > 80% will continue without additional support. 
 
 
 
  
ImPACT + 
PEACE Online 
Resources
Non-Adherent
Continue 
Online 
Resources
Non-Adherent
Add Indiviudal 
Facilitation
Add Group 
Facilitation
Adherent
Continue 
Online 
Resources
Add Group 
Facilitation
Non-Adherent
Add Individual 
Facilitation 
Continue 
Group 
Facilitation
Adherent
Continue 
Online 
Resources
Adherent
Continue 
Online 
Resources
Non-Adherent
Add Individual 
Facilitation
Add Group 
Facilitation
Adherent
Continue 
Online 
Resources
8-Week 
Fidelity 
Assessment 
16-Week 
Fidelity 
Assessment
Randomization 
Exhibit 1: Smart Trial Design 

Project PEACE for ImPACT Research Protocol 
 
Page 5 of 20 
Version 2.0 
Measures 
Providers’ caregiver coaching adherence, the primary outcome, will be assessed using the 
PEACE Caregiver Coaching Fidelity Tool from video recorded observations of community-
based EI sessions. The tool consists of 25 items, rated on a 5-point scale with a rating of 4 or 5 
indicating acceptable adherence. A score of 80% indicates adherence on the overall measure 
(i.e., 80% of the items rated at 4 or 5). Trained research assistants will code caregiver coaching 
adherence from video recordings. We will assess adherence at 8-week intervals for 6 months to 
allow a sufficient length of intervention to observe changes in parent and child outcomes (i.e., 8, 
16, and 24 weeks of intervention).  
 
Acceptability, Appropriateness, and Adoption of the PEACE toolkit will be measured using a 
measure which specifically queries for respondents views about the usefulness, acceptability, and 
the extent to which they used the components of the toolkit. Respondents rate the toolkit’s 
acceptability, appropriateness, and their use of the components on a 5-point scale. We will 
collect this measure after 6 months of intervention from all providers.    
 
Statistical analyses. The design allows for two sets of comparisons, one based on the two 
randomizations, and one based on the embedded support sequences.  
Randomization Comparisons: Our primary comparison concerns the (non-adherent) providers 
randomized at eight weeks, and tests whether providing group support at 8 weeks yielded 
improved fidelity at 16 weeks and 24-weeks. We will compare the groups using a linear mixed-
effects model (Molenberghs & Verbeke, 2001; Verbeke, 1997), with the lower of the provider’s 
two fidelity scores at 16 weeks, and at 24-weeks, as the repeated responses, and randomization 
group (no support vs PEACE group facilitation) as the main explanatory variable. The model 
will also include a binary factor for time (16 vs 24 weeks), and a group by time interaction to 
account for possibly differing effects at weeks 16 and 24.  
Our secondary comparisons will be based on the providers randomized at 16-weeks. 
They will examine the effect of the 16-week randomizations on the fidelity scores at 24 weeks, 
using linear regressions models with the worst 24-week fidelity score as response, and the 16-
week randomization group (PEACE individual facilitation vs group facilitation) as the 
explanatory variable. In further analyses, we will examine the effects of providers’ 8-week status 
(adherent vs non-adherent, group facilitation vs no support) on the 24-week fidelity outcome, by 
extending the models to include 8-week adherence and support (group vs none) as binary factors 
and examine their interactions with 16-week randomization group.  
 
For our primary comparison of the 24-week fidelity responses, we will use a linear regression 
model on this expanded dataset, possibly after transforming the response to reduce skewness. 
The explanatory variable of primary interest will be a four-level categorical indicator of adaptive 
support sequence. The model will provide an overall test of differences in the distribution of 
fidelity score across the four sequences, and provide contrasts between subsets of the sequences. 
Our main interest is in pairwise comparisons between sequence 1, which is a “control” sequence, 
and the other three sequences. 
Power: Based on preliminary data, we anticipate high rates of non-adherence at week 8, so 
differences between sequences will be largely due to differences among the providers 
randomized at eight weeks. We base power calculations on the methods of Seewald et al 

Project PEACE for ImPACT Research Protocol 
 
Page 6 of 20 
Version 2.0 
(Seewald et al., 2020; Formula (10), Design II). If 140 providers are randomized at week 8, and 
that the correlations between fidelity scores within a given provider over time are at least 0.5, 
and that 30% of the providers randomized to continuing online support are adherent at 16 weeks 
compared to 60% of those randomized to group support, then the design provides 80% power to 
detect effect sizes of Cohen’s d=0.52 or higher. 
 
Aim 2: Assess caregiver and child outcomes for families receiving caregiver coaching. 
Research Question 1: Does improved EI provider coaching fidelity lead to improvements in 
caregivers’ responsive interaction strategies with their child?  
Research Question 2: Does improved caregiver use of responsive interaction strategies lead to 
improvements in their child’s social-communication skills?  
Overview 
We will use validated measures of caregiver responsiveness and children’s social communication 
skills to assess changes in caregiver and child outcomes associated with improvements in 
providers’ caregiver coaching fidelity. These measures have been used by the research team in 
similar community-based evaluations of parent-mediated interventions and are sensitive to change 
in caregiver and child outcomes following intervention.   
 
Measures   
 
Caregiver responsiveness will be assessed using the Parenting Interactions with Children: 
Checklist of Observations Linked to Outcomes (PICCOLO: (Roggman et al., 2013). The 
PICCOLO is a checklist of 29 observable, developmentally supportive parenting behaviors with 
children ages 10-47 months; it has been used to effectively measure changes in caregiver 
responsiveness with caregivers of children with autism and developmental disabilities through 
five years of age (Alquraini et al., 2019; Innocenti et al., 2013). The PICCOLO has four 
domains: Affection, Responsiveness, Encouragement, and Teaching. Total scores on each 
domain are used to monitor change in parenting behaviors. It is a strengths-based measure of 
parenting interactions that predicts children’s early social, cognitive, and language development 
with solid psychometrics and has been successfully used to assess changes in parent 
responsiveness with families of children with disabilities and from diverse ethnic backgrounds 
(Roggman et al., 2013; Vilaseca et al., 2019), and was sensitive to changes with large effect sizes 
in a community-based sample of families receiving Project ImPACT (Stahmer et al., 2020). A 
brief (10 minute) parent-child interaction will be video recorded at baseline and after 6 months of 
intervention to assess changes in parent responsiveness; members of the research team blind to 
study arm or time point will score the checklist from the video recorded interaction.   
 
Children’s Social Communication Skills will be measured using the Autism Impact Measure 
(Kanne et al., 2014) and the Social Communication Checklist. The Autism Impact Measure is a 
25-item parent report questionnaire designed to track incremental change in frequency and 
impact of core ASD symptoms (Mazurek, Carlson, Baker-Ericzén, Butter, Norris, & Kanne, 
2020). It has excellent construct validity, test-retest reliability, convergent validity and structural 
validity and is sensitive to change in children’s ASD core symptoms in response to intervention 
(Houghton et al., 2019; Mazurek, Carlson, Baker-Ericzén, Butter, Norris, Barr, et al., 2020; 
Mazurek, Carlson, Baker-Ericzén, Butter, Norris, & Kanne, 2020). Changes in children’s social 

Project PEACE for ImPACT Research Protocol 
 
Page 7 of 20 
Version 2.0 
communication skills will be assessed using the AIM at baseline and after 24 weeks of the 
intervention. The Social Communication Checklist (SCC) is a 47-item parent report checklist that 
is used as part of the Project ImPACT program to help parents set their child’s social 
communication goals and assess child progress. An initial psychometric evaluation suggests that 
it is reliable, sensitive to change after intervention, and strongly related to well-established 
measures of social-communication functioning (Wainer et al., 2017). The SCC will be collected 
at baseline and week 24.  
 
Statistical Analyses. We will address Research Questions 1 and 2 in the context of comparisons 
of embedded sequences. For each question, we may have more than one parent, or more than one 
child, providing responses for a given provider. To account for this nesting of parents and children 
within provider, we will use the weighted regression methods of NeCamp et al., (2017). The 
responses are continuously distributed, so we will use linear models. The explanatory variable of 
primary interest will be a four-level categorical indicator of adaptive support sequence, and the 
baseline version of the response will be included as a covariate.   
Aim 3: Identify provider and family characteristics associated with the need for increasing 
implementation support. 
Research Question 1: Are provider and family characteristics associated with coaching 
fidelity?  
Research Question 2: Among randomized providers, are provider or family characteristics 
associated with response to implementation support?   
Research Question 3: Does the PEACE toolkit engage targeted implementation barriers?   
Overview:  We will gather preliminary data regarding the association between provider and family 
characteristics and provider fidelity. These secondary and exploratory analyses will identify 
potential levers for personalized implementation supports. In addition, we will conduct qualitative 
interviews with providers purposively sampled from each support sequence to assess whether 
PEACE has engaged the intended target implementation barriers (see Exhibit 3).   
 
Measures  
Provider demographics will be measured at baseline through a brief questionnaire that asks 
questions regarding degree, discipline (e.g., instructor, speech therapist), years in the field, 
experience with children with autism and experience with Project ImPACT components.   
 
Parent demographics will be assessed at baseline using 1) a socio-demographic survey that 
includes demographics, family composition, education and financial resources using relevant US 
Census Survey questions, and 2) The Confusion, Hubbub, and Order Scale (CHAOS) (Matheny 
et al., 1995), a validated parent-report measure of disorder and chaos in the home will be used to 
measure home environmental factors that may interfere with the use of caregiver coaching.   
 
Qualitative Interviews will be conducted individually with 20% of providers purposively 
sampled from each randomized support sequence each year upon completion of the PEACE 
toolkit (at week 24). We will adapt the semi-structured interview guide used in our previous 
work to identify the targeted implementation barriers. Standardized probes will be included in 
the interview guide so that consistency across interviews is maintained. The interview protocol 
will allow us to assess how PEACE engaged the targeted implementation barriers. Specifically, 

Project PEACE for ImPACT Research Protocol 
 
Page 8 of 20 
Version 2.0 
we will query around: 1) providers’ perceptions about whether the PEACE toolkit addressed the 
targeted implementation barriers; 2) if providers experienced additional implementation barriers 
that were not adequately addressed by the PEACE toolkit (to inform potential adaptations); and 
3) providers’ perceptions about the appropriateness, benefits, and challenges of the levels of 
implementation support they received. Interviews will be digitally recorded, professionally 
transcribed, and loaded into Nvivo 12.0 software for data management and analysis.   
 
Data Analyses 
Quantitative Statistical Analyses.  
For Research Question 1, the responses will be the provider’s caregiver coaching fidelity score, 
for each child, at week 8, prior to any implementation support. We will use linear mixed effects 
models to accommodate correlations between scores within the same provider. We will include 
the provider’s discipline (e.g., ABA therapist, Speech Therapist, instructor), years of experience, 
experience with ASD, and caseload size, as provider-level explanatory variables. For 
family/child characteristics, we will include the family education, income, composition, and 
chaos score, and the baseline child scores on each of the four subdomains (restrictive behavior, 
language/communication, social reciprocity, atypical behavior) of the Autism Impact Measure.  
 
To address Research Question 2, we focus on the providers who were non-adherent at week 8. 
Our response will be the change in provider’s caregiver coaching fidelity score from 8 weeks 
through 16 and 24 weeks, and we will use the same provider and family/child characteristics 
described in Research Question 1 above. First, we will address the possible moderating effects of 
the characteristics on the effects of the week eight randomization (group support vs continued 
online support). We will use linear mixed effects models, with the week 16 and week 24 change 
scores for each child as repeated responses, and with binary factors for group and week (16 vs 
24). The models will include the provider and family/child characteristics, and the week eight 
caregiver coaching fidelity score, as main effects and as interactions with randomization group. 
To test for possibly differing moderation effects at weeks 16 and 24, we will include group by 
week interactions, and group by week by characteristic interactions. Next, to examine the effects 
of the week 16 randomizations (group vs individual supports), we will focus on the set of doubly 
randomized providers, and use similar models to test for moderating effects of the characteristics 
on the change in fidelity score from week 16 through week 24.   
 
Qualitative & Mixed Methods Data Analysis. We will use qualitative and mixed methods 
analyses to evaluate Research Question 3 and to assess whether PEACE has engaged the 
intended target implementation barriers. We will analyze transcripts using an iterative process 
based upon an integrated approach that incorporates inductive and deductive features (Bradley et 
al., 2007). Both a priori codes (codes that are developed before coding the data) and grounded 
theory codes (codes for ideas that emerge while reading what the respondents actually say) will 
guide codebook development based on a close reading of the first three transcripts. A priori 
attributes of interest include the extent to which providers describe how PEACE impacted their 
attitudes toward caregiver coaching, their perceptions of caregiver coaching norms, and their 
confidence (self-efficacy) in their ability to deliver effective coaching. Twenty percent of 
transcripts will be double coded and we will use the interrater reliability function in NVivo to 
generate Kappa scores to estimate agreement among coders. If there is less than 90% agreement, 
the coders and investigators will discuss discrepancies and resolve them through consensus. 

Project PEACE for ImPACT Research Protocol 
 
Page 9 of 20 
Version 2.0 
After coding, we will read through all codes to examine themes and produce memos of examples 
and commentary. Mixed methods will analyze themes as a function of provider characteristics to 
identify patterns of responding, consistent with best practices in mixed methods research 
(Dowding, 2013).  
Aim 4: Evaluate the cost and cost-effectiveness of tiered implementation strategies.   
Research Question 1: What are the costs and cost-effectiveness (CE) of the PEACE 
implementation toolkit and increasing levels of implementation support? 
 
Overview 
We will conduct two sets of cost and CE comparisons, one based on the two randomizations, and 
one based on the four embedded strategies. Our cost analysis will take a payer (i.e., Philadelphia 
EI system) perspective focusing on programmatic costs and use Time-Driven Activity-Based 
Costing (TDABC), a well-established micro-costing method, based on process mapping (Kaplan 
& Anderson, 2003). This method has been applied to implementation science by co-I Dr. Cidav 
in other work (Cidav et al., 2020, 2021).    
 
Measures 
We will determine total labor and non-labor costs associated with study conditions and 
embedded strategies. For labor costs, first, as per TDABC, we will create process maps in which 
we will discretely outline the activities performed within each randomization group and 
embedded strategy and determine the personnel involved in each activity (e.g., provider, coach). 
Core implementation activities are use of PEACE online library, attending group facilitation 
meetings, attending individual facilitation meetings, and assessing provider fidelity. Second, we 
will determine frequency and average duration of each of these activities during the study period.  
 
For group facilitation meetings, individual facilitation meetings, and provider fidelity 
assessment, frequency, average duration, and personnel involved will be directly observed and 
documented in real time as part of the trial administrative records. For example, during each 
group facilitation meeting, we will have one staff assigned to record date, start and end time of 
the meeting, and who was present. For use of the online resource library, since continuous 
documentation is burdensome and infeasible (e.g., providers recording each virtual 
communication), we will administer activity time logs by which the providers will record their 
time spent on using the online library over a pre-determined period (e.g., staff records virtual 
communication over a randomly selected 1-week period at different time points during the study 
period). Providers will report their online resource library and chat use by a self-report survey. 
 
Once we determine how many times personnel participated in each activity and the average 
duration of the activity, multiplying frequency and duration will yield total time spent by the 
specific personnel on a given activity. Multiplying this by the personnel’s hourly wage rate 
(including benefits) will yield total cost incurred by the personnel to perform that specific 
activity. The total cost of an activity is the sum of costs incurred by each personnel who 
participated in that activity. Adding up the cost of each activity within a specific randomization 
group/embedded strategy will yield the total labor cost for that randomization group/embedded 
strategy. For non-labor resources, we will itemize consumable equipment and supplies (training 
materials, assessment and evaluation materials, office supplies, etc.). Cost of each item will be 

Project PEACE for ImPACT Research Protocol 
 
Page 10 of 20 
Version 2.0 
determined from the project budget and administrative records. Labor and non-labor costs will be 
added together to determine overall costs for each randomization group/strategy.  
 
Statistical Analysis 
Total and per-provider costs for each randomization group and strategy during the study period 
will be determined. We will examine activity, personnel and agency-level variation in costs and 
its determinants. We will conduct sensitivity analyses to estimate the potential costs under 
alternative scenarios where we assume hypothetical modifications (e.g., effects on costs of 
increasing frequency of individual facilitation meetings).   
Finally, we will combine the outcomes of different study arms and embedded strategies 
with their respective costs to provide a measure of relative cost-effectiveness. The effects will be 
improvements in provider implementation fidelity. Incremental cost-effectiveness ratios (ICER) 
will be computed as the ratio of the difference in mean costs (incremental cost) to the difference 
in mean effects (incremental effect). We will compare the ICERs for each randomization 
group/strategy to each other to assess which approach is most cost-effective. Using non-
parametric bootstrapping procedures (Glick et al., 2014), we will construct confidence intervals 
for the ICERs and cost-effectiveness acceptability curves to study cost-effectiveness based on 
different monetary threshold values attached to the improvement in implementation fidelity.   
 
 

Project PEACE for ImPACT Research Protocol 
 
Page 11 of 20 
Version 2.0 
----------------------------------------------------------------------------------- 
PROTECTION OF HUMAN SUBJECTS 
Research will not begin until the Institutional Review Boards of the University of Pennsylvania 
and the City of Philadelphia approve the proposed study. 
 
Characteristics of Population 
Participants will be providers in the Philadelphia EI system, children ages 18 months to 5 years 
being treated by these providers, and their parents. All primary data collection will be conducted 
based on informed consent. Two distinct groups of participants will be recruited in the study (EI 
providers and families). Separate recruitment and consent strategies will occur for each group of 
participants.   
 
Providers: We will recruit 200 providers. All providers have a Bachelor’s or Master’s degree in a 
relevant field (e.g., psychology, education, speech pathology). Inclusion criteria are: 1) employed 
by a Philadelphia EI agency; and 2) have > 5 children with/at risk for ASD on their caseload. 
Based on data from the Philadelphia early intervention system, we anticipate that the sample of 
providers will be 90% female and 10% male. With regard to race/ethnicity, we anticipate the 
following breakdown for providers: 63% Caucasian, 23% African American, 12% 
Hispanic/Latino, 1% Asian, and 1% American Indian/Alaska Native.  
  
Children and Families: We will recruit 400 caregiver and child dyads (2 families per provider). 
Inclusion criteria are that children must: 1) be receiving EI services from a participating 
provider; 2) have a classification of ASD or high ASD risk as determined by the EI system; 3) 
child is not transitioning from EI services for at least 6 months; 4) caregiver speaks English or 
Spanish (Project ImPACT materials are available in Spanish); and 5) have a caregiver willing to 
participate in weekly coaching sessions. Based on current EI family demographics, we anticipate 
the sample of parent-child dyads will be 45% African American, 26% Latino, 18% Caucasian, 
6% Asian, 2% American Indian/Native American, and 3% multi-racial. Children will be less 
than 5 years of age and will either have a diagnosis of autism spectrum disorder or be identified 
as at-risk for autism by the early intervention system, as children under three years of age do not 
need a formal diagnosis to receive autism-specific treatment. We anticipate that 75% of the 
children will be male based on the sex distribution of autism. Participants will be excluded if 
they do not speak English or Spanish, as the published intervention materials are currently 
available in English and Spanish and the research team is able to communicate with participants 
in English or Spanish only. Participants will not be excluded based on their gender or past 
experience.  
 
 
 
 
 
 

Project PEACE for ImPACT Research Protocol 
 
Page 12 of 20 
Version 2.0 
Data to be collected from Human Subjects 
Early Intervention Providers 
Measure 
Baseline 
Week 8 
Week 16 
Week 24 
Demographic 
X 
 
 
 
Fidelity 
Observation 
 
 
X 
X 
X 
Acceptability and 
Feasibility 
 
 
 
X 
Qualitative 
Interview 
 
 
 
X 
Parents 
Demographics 
X 
 
 
 
CHAOS 
X 
 
 
 
Autism Impact 
Measure 
X 
 
 
X 
Social 
Communication 
Checklist 
X 
 
 
X 
10 minute 
recorded play 
interaction 
X 
 
 
X 
 
Data will take the form of demographic questionnaires, direct observations, and audio and video 
recordings. Data will be obtained directly from participants in the proposed research projects. No 
data will be obtained from existing records.   
Field Observations: We will collect direct observation ratings of provider’s coaching fidelity 
using the Peace Caregiver Coaching Fidelity Tool and the Project ImPACT Fidelity Checklist. 
Session recordings will be of usual early intervention sessions, approximately 1 hour long. We 
will record 3 sessions with each family enrolled in the study (at 8 weeks, 16 weeks, and 24 
weeks of intervention).  
Demographic Questionnaires: We will collect sociodemographic information for each participant 
including information related to race/ethnicity, socioeconomic status, and education level at 
baseline.   
Questionnaires: We will collect parent surveys regarding the home environment using the 
Confusion, Hubbub, and Order Scale (CHAOS), and surveys regarding children’s social 
communication skills using the Autism Impact Measure and the Social Communication Checklist 
(SCC). Parents will complete the CHAOS scale at baseline and the Autism Impact Measure and 
the Social Communication Checklist at baseline and after 24 weeks of intervention. We will 
survey providers on the acceptability, appropriateness and feasibility of the PEACE 
Implementation Toolkit at 24 weeks of intervention.   

Project PEACE for ImPACT Research Protocol 
 
Page 13 of 20 
Version 2.0 
Assessments: We will collect direct observation ratings of caregiver responsiveness using the 
Parenting Interactions with Children: Checklist of Observations Linked to Outcomes 
(PICCOLO) from 10-minute recordings of a parent-child play interaction at baseline and after 24 
weeks of intervention. 
Interviews: We will conduct qualitative interviews with 20% of the early intervention providers 
in each cohort. We will interview them using an interview guide with questions eliciting 
feedback on the PEACE Implementation Toolkit and the implementation supports that were 
provided. Interviews will be recorded on Zoom for audio or over the phone based on participant 
preference. 
Use of Toolkit: We will gather data on utilization of the online resource library and chat by self 
report by provider participants. During a randomly selected week of the intervention, providers 
will report the amount of time spent accessing and utilizing the online resources. 
Recruitment and Informed Consent 
Recruitment 
Clinicians 
Clinician recruitment will occur through participating agencies. We will provide information about 
the study to agencies participating in Philadelphia’s Project ImPACT training initiative and invite 
them to participate. Recruitment and retention strategies are based on our prior community trials 
in which we recruited 80-95% of eligible practitioners and experienced 5-7% attrition. Agencies 
will distribute flyers and a short video describing the study to EI providers employed in their 
agency. Agency leaders will also describe the study at staff meetings. Research team members will 
attend professional development and informational meetings to answer providers’ questions, 
address their concerns and garner support. If a provider agrees to participate, they will speak with 
study staff over the phone to discuss the study and review the consent form. The consent form will 
be provided to the provider to sign either through an electronic link or mailed to them, based on 
their preference. There will be a local number and e-mail address for staff to contact with any 
study-related questions. 
Families 
After a provider has agreed to participate in the study, agencies will then distribute flyers and a 
short video to parents on the providers’ caseload describing the study. Interested parents will sign 
a permission to contact form using a secure electronic survey link. Initial contact with families will 
be made by telephone during which eligibility will be confirmed, the study will be described in 
detail utilizing the consent form, and all questions about the study will be answered. A secure 
electronic link to the study consent form will be emailed or texted to the parent to review and sign. 
If a parent prefers and requires it, the consent form can also be mailed to the parent to sign. Once 
the signed consent form is received, a member of the research team will begin data collection. 
Consent will be obtained from both providers and families before data collection begins.  There 
will be a local number for families to call with any questions related to the study protocol. 
 
Consent Process 
The amount and complexity of data to be collected warrants affirmation of consent from study 
participants at several steps in the process. The entire study will be described during the 

Project PEACE for ImPACT Research Protocol 
 
Page 14 of 20 
Version 2.0 
invitation phone call to the provider or parent who has indicated interest in the study. Interest in 
moving on to the consent process will be obtained verbally at this time. Informed consent will be 
obtained electronically through a link sent to the provider or parent or on paper via a mailed 
consent form. The provider or parent will be able to review the consent form and sign if they 
agree to participate. In addition, informed consent will be affirmed at each step of data 
collection. The participant will have the opportunity to ask questions about the consent forms 
prior to giving written informed consent.   
 
A written informed consent for the study will be sent in an electronic email or mail for participant’s 
review prior to the start of data collection. The document and link includes a description of the 
entire study and expected roles and responsibilities of the provider, primary caregiver and child, 
risks and benefits to participants, and confidentiality procedures. The consent document will also 
clearly state that participation in the study is completely voluntary, and participants can drop out 
of the study without penalty at any time. Study staff will review this consent form over the phone. 
At this time, study staff will answer questions and obtain verbal confirmation of interest to review 
the consent form and sign through the electronic link, or that they would like the consent form 
mailed to them with a return envelope. The consent forms explicitly outline the voluntary nature 
of participation in this research study. Easy channels of communication to study staff if a 
participant wishes not to participate will be clearly outlined. Specifically, the contact information 
for the PI and study coordinator will be listed on the consent form, expectations for participation, 
compensation, and further outlines of confidentiality practices specific to information collected. 
All participants will be given the option to consent to receiving text messages from the research 
study team. Text messages will be used for sending study updates and confirming study procedure 
appointments. Text messages will be sent from the  research study team’s email. Participants will 
be informed that standard text message rates apply if they agree to allowing the research team to 
contact them via text messages.  
All consent documents follow the guidelines outlined by the Office of Human Subjects Protection 
of the National Institutes of Health, the University of Pennsylvania, the Philadelphia Department 
of Public Health, and ethical guidelines set forth by the state and federal governments. 
Documentation of subject consent will be stored in a locked file in the PI or designee’s locked 
office.   
 
Subject Compensation 
Participating providers and families will be compensated for time spent completing specific data 
collection time points.  
 
Field Observations and Survey Completion: Providers will be compensated $25 for participating 
in each fidelity observation. This will be a total of 6 observations (3 time points with 2 families), 
for a total of $150. Families will be compensated $75 for participating in the parent-child play 
interaction observations and completing questionnaires at two time points for a total of $150.   
 
Professional Development: Providers will receive professional development training credits for 
their participation in group facilitation meetings and for completing the training. In addition, they 
will receive training in Project ImPACT and the option to receive certification in Project ImPACT 
at no cost, which is a considerable cost benefit for the provider.  
 

Project PEACE for ImPACT Research Protocol 
 
Page 15 of 20 
Version 2.0 
Data Sharing 
 
If providers believe that their performance in the study may be evaluated by their agency or the 
EI system they may become anxious and thus make them unlikely to participate in the study. 
Videotaping provides concrete examples of coaching techniques that enable efficient and 
accurate measurement of the strategies used during intervention sessions; however, there is 
potential that such videotaping will provide concrete examples of intervention techniques or 
other behavior that could negatively influence staff performance review if administrators or 
supervisors in the EI system were to view it. For this reason, there will be no disclosure of 
videotaped performance or self-reported information about provider fidelity, especially to these 
personnel. As a result, the research team will keep all identifiable and individual participant data 
confidential from all administrative personnel, including agency executive and clinical directors. 
However, upon conclusion of data analysis, the research team will provide anonymous and 
aggregate results of the study to the Philadelphia Department of Public Health, the department of 
Intellectual and Disability Services within DPH, Philadelphia Infant and Toddler Early 
Intervention, Elwyn Preschool Early Intervention, as well as any participating community 
agencies. Anonymous and aggregate results provided to community agencies will be aggregate 
data from all participants, not exclusively those from any single community agency. This 
procedure acts in the interest of participant confidentiality and job security. 
Potential Risks to Subjects 
 
Risk to parents and children  
Parent and child involvement in the proposed study involves direct observation measures of 
children and parents in their homes, completion of a demographic survey, and completion of 
surveys regarding the child’s social communication skills. The study therefore involves minimal 
risk to children and their families. The risks include:  
1. 
Respondent burden associated with the time and effort to complete the 
observations and surveys:  
a. 
We estimate time burden to be an average of 3.5 hours of total data 
collection for parents during observations (10-minute parent-child 
interaction observation at 2 time points, 1- hour fidelity observation at 3 
time points).  
b. 
We estimate time burden to be an average of 30 minutes for survey 
completion at 2 time points.  
2. 
The possibility of breached confidentiality by disclosure of personal 
information.  
3. 
The risk of distress caused by the videotaped observation and/or 
questionnaires:  
a. 
The proposed measures include a number of sensitive items related 
to children’s clinical presentation and family characteristics. Answering 
certain questions on the proposed measures may cause some distress. 
Some questions asked of parents (such as those about income or stress) 
may cause parents some discomfort.  

Project PEACE for ImPACT Research Protocol 
 
Page 16 of 20 
Version 2.0 
b. 
Parents may feel anxious by observations or videotaping, 
especially if they believe that the research team is evaluating their 
performance. Videotaping provides concrete examples of teaching 
techniques that are vital to the measurements proposed in this research. 
However, there is potential that such videotaping will provide examples of 
behavior that the research team believes endanger the parent or child. For 
this reason, there will be no disclosure of observed or videotaped 
information to those outside of the research team unless there is a 
suspicion of child abuse or neglect. Parents will be informed during the 
consent process that all study personnel are mandated reporters. All staff 
are trained in mandated reporting. In cases of suspected abuse or neglect, 
the research staff member suspecting the abuse or neglect will contact one 
of the two licensed psychologists on the study. They will review the 
situation together and determine if it meets criteria for reporting.  
  
4. 
The risk that the intervention may have some iatrogenic effect.  
a. 
There is a small chance that the proposed intervention may have 
negative consequences. The majority of evidence suggests that the 
components of the proposed intervention are the most efficacious in 
treating young children with ASD. However, to protect against this risk, 
we will monitor outcomes and potential adverse events, as described 
below, to identify any iatrogenesis.  
 
  
Risk to Early Intervention Providers  
Early intervention providers’ involvement in the proposed study involves direct observation 
measures of intervention fidelity, completion of 1 sociodemographic survey, brief measures on 
the acceptability and appropriateness of the toolkit, and participation in qualitative interviews. 
The study therefore involves minimal risk to providers. Possible risks include:  
1. Respondent burden associated with the time and effort to complete the training, 
observation, interviews, and questionnaires:  
a. 
We estimate time burden to be an average of 22 hours for completing 
Project ImPACT training before baseline. 
b. 
We estimate time burden to be an average of up to 16 hours for 
participation in group facilitation meetings, if providers are assigned to the 
implementation group. 
c. 
We estimate time burden to be an average of up to 8 hours for 
participation in individual facilitation meetings, if providers are assigned to the 
implementation group. 
d. 
We estimate time burden to be an average of 6 hours of total data 
collection for providers during observations (1 hour observation at 3 time points 
with 2 families).  
b. 
We estimate time burden to be an average of 15 minutes for survey 
completion at baseline and 24 weeks.  

Project PEACE for ImPACT Research Protocol 
 
Page 17 of 20 
Version 2.0 
2. The possibility of breached confidentiality by disclosure of personal information 
to administrative personnel at the early intervention system linking individual 
providers to their performance.  
3. The risk of distress caused by the videotaped observation, interview content, 
and/or questionnaires:  
a. 
Providers may be made anxious by videoing, especially if they believe that 
their performance in the study may be evaluated by their agency or the EI system. 
Videotaping provides concrete examples of coaching techniques that are vital to 
the measurements proposed in this research; however, there is potential that such 
videotaping will provide concrete examples of intervention techniques or other 
behavior that could negatively influence staff performance review if 
administrators or supervisors in the EI system were to view it. For this reason, 
there will be no disclosure of videotaped performance or self-reported information 
about provider fidelity, especially to these personnel. To help minimize this risk, 
we will provide EI administrative personnel, including agency executive and 
clinical directors, written notification that these measures are confidential and at 
no time and under no circumstances will be shared with EI personnel.   
b. 
Potentially more distressing are emotional responses to fidelity measures. 
The fidelity measures are designed to quantify EI providers’ adherence to the 
intervention protocol. EI providers may feel as though their willingness or ability 
to treat children with ASD is being challenged. Specifically, documenting 
providers’ performance may cause feelings of insecurity regarding effort or 
ability.  
 
Protection against Risks 
Interaction with research participants. To minimize distress associated with respondent burden and 
distress associated with the interview, both parents and providers will be told at the beginning of 
each observation or have stated at the beginning of the questionnaire that they may refuse to 
participate in any part of the study and skip any question at any time. We will accommodate 
parents’ wishes with regards to the timing of the observations, will break up data collection into 
stages if requested, and will train research staff to be sensitive to the well-being of parents, 
children, and providers. Participants will be reminded that information will be kept confidential 
and will never be published in any manner that would identify them. All participating families will 
be reminded that all clinical data will be collected for research purposes only. Children’s distress 
is most likely to arise from stress or fatigue associated with observations. Frequent breaks will be 
offered as well as the ability to reschedule for a more optimal time. Families will be encouraged 
to contact their pediatrician, school evaluation center, or local developmental specialists if they 
desire further evaluation.  
Data Management and Access. To minimize risks associated with breaches of confidentiality, all 
data pertaining to study participants will remain confidential at all times. The exposure of the 
identity of study participants will be avoided wherever possible. An 11-digit identification number 
will be employed to encode the participant identity on study materials, including data collection 
forms. Providers will be assigned a unique identification number as well. The Penn Center for 

Project PEACE for ImPACT Research Protocol 
 
Page 18 of 20 
Version 2.0 
Mental Health at the University of Pennsylvania, where data will be stored, has a 20-year history 
of conducting research using protected health information and linking it with other datasets.  
Center protocols for data management and protection have been developed and implemented for 
all datasets that include protected health information. Data records will be labeled confidential, 
use only unique identification numbers and be kept on a secure server that is only accessed by 
those on the study team. The research team will handle the original media on which data are kept. 
Media will be labeled confidential, use only unique identification numbers and be kept in a locked 
filing cabinet. Penn’s Technology Support Services employs a number of security and privacy 
technologies and best practices in order to deliver services that provide “security in depth”.  
We plan to maintain access to the video and audio recorded data of participant field observations 
and interviews for approximately 3 years (or until analyses are complete) from the end of data 
collection. We will ask participants that consent to the study to sign a video release form to indicate 
whether they will allow us to keep video files beyond the 3 year expiration date for training and 
presentation purposes. Data will be maintained for the purpose of coding and analysis to inform 
the primary aims of the study.   
 
To ensure the privacy and confidentiality of data for this project we will only store and use the 
identifiable data at the following locations: 1) Password-protected PCs; 2) A server at the 
University of Pennsylvania within the facilities managed room in the data center. A Senior LAN 
Consultant will upload the data onto the secure production servers. Videos will be sent from Penn 
to Dr. Ingersoll’s lab at Michigan State University for coding. Videos will be sent via Penn Box, 
Penn’s secure file sharing service. When viewing videos at MSU, staff will keep the video copy 
in Penn Box and not make copies to other media.  Lastly, all output containing individual 
identifiable information is treated as confidential data. This information is never transferred 
electronically via email or other protocols. Any printed material containing individual identifiers 
is shredded.  
Physical Security – Servers are hosted in dedicated virtual machines running on Dell SAN 
hardware in the Information Systems and computing facilities managed computer room. The 
hosting environment SAN and Host Bus Adapters are in a locked Dell rack housed in the Computer 
Room with restricted door card access to authorized personnel only. There is 24-hour camera 
surveillance. 
Business Continuity – Server environments get redundant power from independent power feeds. 
In addition, each of the power sources is UPS protected. There is a Halon fire suppression system, 
with alarm points below the raised floor and in the ceiling. The virtual machines are hosted in 
VMWare ESX environments connected to a Dell SAN for shared storage. All virtual machines are 
stored on the SAN. To minimize downtime caused by hardware failure, the environment is 
configured for redundancy with multiple ESX hosts, multiple Host Bus Adapters, multiple SAN 
Switch ports, multiple storage processors, and RAID 5 storage. The server environments are 
replicated to a secondary site at 3650 Chestnut Street, which has equivalent physical security and 
also employs all of the security best practices outlined below. 
Server Administration and Maintenance – The servers are administered by a team of four full-time 
professional IT staff. The team has over 50 years of combined experience configuring and 

Project PEACE for ImPACT Research Protocol 
 
Page 19 of 20 
Version 2.0 
supporting Windows servers, and all of the staff attend technical training regularly to stay current 
in best practices for configuring and securing Windows and VMWare environments. 
Server Security Best Practices – In keeping with SANS and Microsoft best practices, all software 
services and corresponding ports on the servers that are known to be substantial security risks and 
which are not used by CMH resources are disabled including telnet, and ftp. Security patches are 
applied promptly and there are standard processes in place for preventive maintenance and 
monitoring of the servers. 
Hardware Firewalls - All servers reside behind a cluster of Juniper SSG520 firewalls. A Juniper 
SSG520 appliance configured in high availability mode acts as a secure gateway between 
PennNet/Internet (Untrusted Zone) and server resources behind the firewall. A custom firewall 
policy is developed for each resource hosted on each of the virtual machines. All policies are 
developed based on service port and PennNet only IP addresses.  
VLAN - A Foundry Switch in the TSS managed rack is configured for 4 VLANs to support traffic 
segmentation behind the SSG520. LTS has access to each VLAN, giving us the ability to place 
each virtual machine in the appropriate VLAN for another layer of security. 
SSL Encryption for Web Services - When hosting web servers we only support the use of HTTPS 
with an SSL certificate to minimize vulnerabilities and exploits common on the standard HTTP 
port. All port 80 traffic is re-directed to port 443. 
Password Policy - A complex password policy, meeting Microsoft complex password 
requirements, is in effect in the CMH domain. The password policy requires all CMH domain 
users to change their domain password every 180 days. 
Preparation of Staff for Data Collection 
All study personnel will have Patient Oriented Research certification from the University of 
Pennsylvania. The PI will oversee the additional training and ongoing review of interviewers and 
coders. Clinical research staff who oversee assessments will be trained to reliability and supervised 
by the clinical psychologist on staff. In order to become reliable, staff will score videotaped 
treatment sessions. The PI will then score the same videotaped sessions and the PI’s results will 
be compared with the examiner’s results. The PI will calculate inter-rater reliability rates; inter-
rater reliability should be no less than 80%. The PI will also provide all research assistants with 
training regarding culturally sensitive practices and maintaining professional boundaries while 
interacting with families.   
Adverse Events 
All unexpected adverse events will be reported by study personnel to the Philadelphia Department 
of Public Health IRB, the Project Officer at IES, and the IRB at Penn via standard adverse event 
reporting procedures. All adverse reactions will be noted and discussed thoroughly with the PI, 
investigative team, and IRB to determine proper reporting procedures on a case-by-case basis. 
Potential Benefits of the Proposed Research to the Participants and Others  

Project PEACE for ImPACT Research Protocol 
 
Page 20 of 20 
Version 2.0 
There are some benefits to family/child and clinician participants. Families in the treatment 
conditions will receive training in intervention techniques considered best practice for young 
children with ASD. In addition, clinicians in the two treatment conditions will receive considerable 
training and support throughout their participation that has the potential to benefit the children they 
treat. The potential exists for great benefits to child-serving systems at large. Potential risks are 
reasonable given the safeguards proposed and the valuable information to be yielded regarding 
interventions for children with ASD.