A Randomized Trial of Project ImPACT, an Evidence-based Intervention for Infants and Toddlers With Autism Spectrum Disorder
Phase: NA
Conditions: Autism Spectrum Disorder
Sponsor: University of Pennsylvania

A RANDOMIZED TRIAL OF 
PROJECT IMPACT, AN 
EVIDENCE-BASED 
INTERVENTION FOR INFANTS 
AND TODDLERS WITH AUTISM 
SPECTRUM DISORDER 
Principal Investigator: Melanie Pellecchia, PhD 
Version 6: 3/1/2021 
IRB Protocol #842506; NCT04729127 

Project ImPACT Research Protocol 
 
Page 1 of 18 
Version 6 
 
 A Randomized Trial of Project ImPACT, an evidence-based intervention for 
infants and toddlers with autism spectrum disorder  
RESEARCH PROTOCOL 
 
ABSTRACT 
One in 59 children has ASD; approximately 40% are diagnosed before three years of age.2 The 
Individuals with Disabilities Education Act (IDEA)17 entitles these children to services that 
address their delays and prepare them for school. Evidence-based treatments for infants and 
toddlers with ASD include a therapist-implemented component and parent training, with the 
expectation that parents will continue the intervention between therapist visits. There is growing 
interest in the effects of parent-mediated intervention for young children with ASD,18 driven by 
the belief that empowering parents to use intervention strategies within daily routines is more 
developmentally appropriate than intensive, therapist-led interventions.11 High-quality, parent-
mediated EI improves long-term outcomes for children with ASD across a range of developmental 
domains.7,16,19 It also improves parental self-efficacy and treatment engagement.7,20,21 Several 
efficacious parent-mediated interventions have been developed in research settings. Beyond the 
question of efficacy – the extent to which these interventions achieve outcomes under ideal 
circumstances – there has been a growing call to test their effectiveness – the extent to which they 
work under real-world circumstances.9,10 Researchers have pointed to the ‘voltage drop’ that 
occurs when evidence-based interventions move from the lab to the community,22 and the need to 
test whether they are successful when community clinicians use them.23-25 The World Health 
Organization bemoans the lack of effectiveness trials, which inhibits its ability to strongly 
recommend parent training.26 While Project ImPACT is designed for community implementation, 
it has not been subject to a rigorous effectiveness trial. 
Services in publicly-funded EI often are limited to 1-to-3 hours a week of therapist contact. 
Little data are available on how many contact hours are necessary to change parent behavior, 
produce positive child outcomes, or the associated cost-benefit. One small, non-randomized study 
found that a low dose of Project ImPACT was enough to improve parent-reported outcomes 
relative to the comparison group, but the sample size, lack of randomization, use of highly-trained 
academic clinicians, and lack of a comparison group at another dose of Project ImPACT limits 
interpretation of these findings.19 In addition to providing data on effectiveness, our study also will 
estimate cost-effectiveness. There is little evidence about cost-effectiveness, which is needed to 
support resource allocation decisions.27 While several studies simulate data to suggest that the 
costs of early intervention are offset by long-term savings,28-31 only two studies comprise a formal, 
rigorous cost-effectiveness evaluation.32,33 None has examined the relative benefit of an 
intervention at different doses, a critically important question to our community partners. In 
Philadelphia, the infant and toddler early intervention program spends $10 million annually for 
services for children with ASD. In the US, these services typically are funded with public dollars. 
It is incumbent upon service systems to support interventions that produce meaningful outcomes 
and to avoid the use of public funding for interventions below or above an effective dose. With 
better information about effective dose of intervention, service systems can more effectively direct 
and allocate public dollars. 
Our pilot study will address a number of pressing questions, including: Can community 
clinicians successfully coach parents of children with ASD in evidence-based practice to achieve 
positive outcomes for their children and themselves? What dose is necessary to achieve change? 

Project ImPACT Research Protocol 
 
Page 2 of 18 
Version 6 
And what is the cost effectiveness of implementing evidence-based interventions at different 
doses? This project has the potential to dramatically improve the services families of young 
children with ASD in Philadelphia receive, and significantly improve long-term outcomes. 
Successful completion of the proposed study would provide the EI system with the information 
needed to bring this type of intervention to scale across the entire system. Findings from this pilot 
trial would serve as the foundation for an application for federal funds to conduct a large-scale, 
system-wide study of Project ImPACT implemented in all Philadelphia agencies serving infants 
and toddlers with ASD. 
 
RESEARCH PLAN 
The proposed study is highly innovative. Foremost, the choice of intervention, the study questions 
and the study design all were developed in partnership with leaders from Philadelphia’s infant and 
toddler early intervention system, which increases the ecological validity and utility of the 
findings, and the potential for the study to lead to lasting changes in Philadelphia’s early 
intervention system.34 Second, the study comprises a field trial of an intervention for young 
children with ASD using community (as opposed to university-based) clinicians to train parents in 
the intervention. This model of testing effectiveness is rare, with similar studies published mostly 
from data collected outside of the United States.15 Third, our sample will be culturally and socio-
economically diverse, a limitation of almost all prior autism intervention studies.10 Fourth, we 
propose to test two doses of the same intervention. While determining the most effective dose of 
intervention is of critical importance to the field, there has been no head-to-head comparison in a 
randomized trial. Fifth, we propose to measure family as well as child outcomes, which rarely have 
been reported in intervention trials.35  
 
Setting for all Research Activities  
In Philadelphia, children ≤36 months of age with ASD are eligible for treatment through the infant 
and toddler early intervention program. Risk is determined using the Modified Checklist for 
Autism in Toddlers.39 Thirteen agencies provide these services. They use an interdisciplinary 
treatment approach, have a treatment philosophy that supports parent-mediated intervention, and 
provide home-based services. The early intervention system projects that >800 children with 
autism ≤36 months of age will receive services next year. On average, children receive 3.4 
hours/week of services, about one hour of which comprises “specialized instruction,” which 
Project ImPACT will replace. Children will retain all other services. In specialized instruction, 
therapists: 1) design learning environments and activities that promote skill acquisition; 2) plan 
activities that address the outcomes on the individualized family service plan; 3) provide families 
with information, skills and support to enhance child skill development; and 4) work with the child 
and family to enhance the child’s development. Specialized instructors typically are bachelors-
prepared individuals with early child development training. 
 
Aim 1: Compare outcomes in young children (social communication and development) and 
their parents (stress and self-efficacy) randomized to one of three groups: A) treatment as 
usual, B) Project ImPACT delivered at 1 hour/week and C) Project ImPACT delivered at 4 
hours/week.  
 
 

Project ImPACT Research Protocol 
 
Page 3 of 18 
Version 6 
Overview 
We will compare the outcomes of young children and their parents randomized to three groups: 
A) treatment as usual, B) 6 months of Project ImPACT at 1 hour/week and C) 6 months of Project 
ImPACT at 4 hours/week. Observations of regular early intervention sessions between families 
and clinicians will provide evidence of adherence to the Project ImPACT intervention. 
Observations of parent and child interactions as well as direct assessments with enrolled children 
over the course of 6 months, will give information on changes in social communication and parent 
responsiveness and self-efficacy. 
Sample  
Clinicians. We will recruit 3 clinicians from each of the 6 participating agencies (18 total). 
Agencies employ an average of 43 clinicians (range 21-64). Clinicians employed in these agencies 
have a Bachelor’s or Master’s degree in a related field, such as psychology, education, or speech 
pathology. Inclusion criteria are that the clinician: 1) has ≥ 5 children with ASD on their caseload; 
2) agrees to participate in training and consultation if selected for one of the Project ImPACT arms; 
3) identifies and obtains consent-to-contact forms for families on his or her caseload; and 4) 
completes the clinician study measures.  
Children and their families. We will recruit 3 children per clinician (totaling 54 children, 54 
parents). Inclusion criteria are that children: 1) be < 30 months of age (so that they can receive ≥ 
6 months of intervention; 2) have a classification of autism or high autism risk as determined by 
the EI system; 3) be receiving early intervention services through the infant and toddler program 
and be participating in telehealth sessions if applicable; and 4) have a parent willing to complete 
the parent measures that are part of this study.  
Setting 
Assessments and fidelity observations will occur in the family’s home, the usual provision of 
family-based early intervention services. The 2 day ImPACT training workshop will take place at 
a mutually desired upon location in Philadelphia convenient for clinicians to attend. Ongoing 
supervision and consultation will take place on video conferencing and phone calls.  
Changes in response to COVID-19: The early intervention system is providing all direct service 
to families via tele-intervention. The research team will not visit families’ home for any research 
activities until local and state authorities indicate that in-home intervention can be resumed. All 
research activities will be conducted remotely via tele-intervention, shared video recordings, and 
electronic surveys. Consistent with all services being delivered by the early intervention system, 
Project ImPACT and treatment as usual will be delivered via tele-intervention. The research team 
will join tele-intervention sessions for fidelity monitoring and coaching. Direct observation 
assessments will now be conducted via observations of self-recorded parent-child play 
interactions. Per these changes, families should have the ability and technology to participate in 
tele-health sessions with their EI provider in order to participate in the study. 
Measures 
Child Outcomes (all measures collected at baseline and 6 months): 
Changes in Social Communication will be measured using the communication and social domains 
of the Vineland Adaptive Behavior Scales–3rd Edition43 and the Preschool Language Scales- 5th 
Edition (PLS). The Vineland is a standardized parent report survey of adaptive functioning that 
has been normed on individuals ages birth to 90 years. It has excellent psychometric properties 

Project ImPACT Research Protocol 
 
Page 4 of 18 
Version 6 
and has been used in many studies of children with autism. Previous research has shown that 
Communication and Social Domain standard scores are sensitive to change in response to 
ImPACT.19,44 The PLS is an interactive direct assessment of developmental language skills that 
can be given to children birth to seven years. Primary domains include auditory comprehension 
and expressive communication. It is designed to test a full range of communication skills across 
the developmental spectrum (including attention, play, gesture, and social communication). It has 
demonstrated strong reliability and validity across a diverse population, including clinical samples 
and children with ASD. We will use standard scores on auditory comprehension and expressive 
communication, as well as total standard scores, in the analyses. A research team member trained 
to reliability of administration will administer the PLS under the supervision of a licensed 
psychologist.  
Changes in response to COVID-19: The Vineland will be administered to parents via a secure 
electronic survey link, instead of in person.  
Children’s Social Communication will be measured using the Brief Observation of Social 
Communication Change (BOSCC) scoring applied to the parent-child interaction videos, instead 
of the Preschool Language Scales. The BOSCC is an observational coding scheme designed to be 
sensitive to changes in social communication behavior, easily used by naïve, minimally trained 
examiners and coded relatively quickly. The BOSCC has strong interrater and test-retest 
reliability, sensitivity to change, and evidence of convergent and discriminative validity. The 
BOSCC scoring procedure will be used to measure child social communication behaviors during 
a parent-child play interaction. The BOSCC does not require direct contact with the parent or child 
to administer, so is an appropriate outcome measure to use in response to the COVID-19 pandemic. 
Parents will self- record a 10-minute play interaction between the parent and child using their 
phone, tablet, or any personal device and will share the recording with the research team via a 
secure Penn Medicine ShareFile link. ShareFile is a Citrix program that allows its users to share 
and receive large files securely.   
Parent Outcomes (All measures will be completed at baseline, 3 months, and 6 months) 
Parenting Self-Efficacy will be measured using the Early Intervention Parenting Self-Efficacy 
Scale (EIPSES),45 a 16-item self-report measure that examines parents’ beliefs about their ability 
to produce positive changes and promote optimal development in their child with a disability. It 
measures two dimensions of self-efficacy: Parent Outcome Expectations and Parent Competence. 
Parents rate items from 1 (“Strongly disagree”) to 7 (“Strongly agree”) with higher scores 
indicative of higher parenting self-efficacy. The EIPSES has strong psychometric properties and 
been used in a number of studies of families receiving Part C services.  
Parent Quality of Life will be measured using the World Health Organization WOL Assessment-
Brief (WHOQOL-BREF), a short version of the WHOQOL-100 scales. The 26-item questionnaire 
covers the four dimensions of QOL (physical, psychological, environmental, and social domains). 
The instrument measures the degree to which the respondent subjectively evaluates their 
satisfaction with different aspects of their life from a score ranging between 1 (very 
dissatisfied/very poor) and 5 (very satisfied/very good).  
 
Changes in response to COVID-19: The EIPSES and WHOQOL-BREF will be administered 
electronically via a link unique to each participant sent via RedCap. 
 

Project ImPACT Research Protocol 
 
Page 5 of 18 
Version 6 
Parent Responsiveness will be measured using the Parenting Interactions with Children: Checklist 
of Observations Linked to Outcomes (PICCOLO),46 a checklist of 29 observable, developmentally 
supportive parenting behaviors with children ages 10-47 months. The PICCOLO has four domains: 
Affection, Responsiveness, Encouragement, and Teaching. It is a strengths-based measure of 
parenting interactions that predicts children’s early social, cognitive, and language development 
with solid psychometrics. A brief (10 minute) parent-child interaction will be video recorded (see 
below) at each time point; members of the research team blind to study arm or time point will 
score the checklist. Total scores on each domain are used to monitor change in parenting behaviors.  
Parent Treatment Adherence will be assessed through a videotaped parent-child interaction at 
home. Parents will be asked to: 1) interact with their child for 10 minutes in the way they typically 
would during play; and 2) have a snack or meal with their child. Parent behavior will be scored for 
use of the intervention strategies using the Project ImPACT Intervention Fidelity Checklist.1 Each 
of 6 strategies is scored on a scale of 1 (“Parent does not implement throughout session”) to 5 
(“Parent implements throughout session), and then averaged to form an overall fidelity rating for 
each routine. Ratings for the play and snack routines will be averaged to form an overall fidelity 
rating. Several studies have used the Project ImPACT fidelity checklist, which shows strong 
interrater reliability.19,20,44 Trained observers blind to group assignment and time point will score 
it. 
COVID-19 Changes: Parents will self-record a 10-minute play interaction and a snack routine with 
their child using their phone, tablet, or other personal device., instead of having a research team 
member record the session during a home visit. The same 10-minute play interaction will be used 
to code for changes in children’s social communication as described above for the BOSCC, 
changes in parent responsiveness for the PICCOLO, and changes in parent treatment adherence 
(using the Project ImPACT Intervention Fidelity Checklist) for each time point.  Parents will share 
the video with the research team using a secure Sharefile link.  
Other Important Covariates will include 1) parents’ socio-demographic characteristics. Parents 
will complete a form that includes demographics, family composition, education and financial 
resources using the relevant US Census Survey questions; and 2) clinician training and experience, 
which will be measured through a brief questionnaire that asks questions regarding degree, years 
in the field, experience with children with autism and experience with Project ImPACT 
components. 
COVID-19 changes: The socio-demographic survey will be administered to parents via a unique 
secure electronic Redcap survey link. 
Procedure 
Clinician training  
Dr. Ingersoll and the MSU research team will train community clinicians assigned to the Project 
ImPACT groups using a modified version of the published provider preparation protocol.40 This 
25-hour training will consist of a 6-hour, self-paced online tutorial, followed by a 2-day interactive 
workshop that covers the program’s rationale and critical elements of the intervention techniques.  
 
COVID-19 changes: The interactive workshop will be delivered to clinicians remotely via Zoom 
instead of in-person. The workshop will be delivered over 4 days, in 3.5 hour sessions, instead of 
2 full days to accommodate the remote learning experience. 
  

Project ImPACT Research Protocol 
 
Page 6 of 18 
Version 6 
Following introductory training, clinicians in each agency will participate in bi-weekly group 
consultation meetings with Penn consultants (see below) via videoconferencing. During these 1-
hour group meetings, clinicians will present their Project ImPACT cases to the other clinicians in 
their agency and receive feedback and problem solving from the Penn consultant. Each clinician 
also will receive a monthly consultation visit in the field. During these 1-hour sessions, the 
consultant will observe an intervention session, complete the coaching fidelity form, and provide 
feedback to the clinician on Project ImPACT implementation. Dr. Ingersoll’s team has 
successfully used this remote consultation model to train clinicians to fidelity in Project ImPACT 
with high ratings of satisfaction, self-efficacy, and attendance from clinicians who participated in 
the training.41 
To prepare Penn staff to consult in the field, Dr. Ingersoll and the MSU research team will 
train them to fidelity in Project ImPACT and its coaching model, and fidelity monitoring. This 
training will include the same introductory training described above, consultation meetings with 
MSU study staff to provide feedback on program implementation with a family with ASD via 
videoconferencing; and obtaining reliability on parent treatment adherence and coaching fidelity 
scoring for master session tapes. Once Penn staff have been trained to fidelity in coaching and 
reliability in fidelity monitoring, they will meet biweekly with MSU staff via video-conferencing 
for supervision as they provide monthly consultation sessions to community clinicians using 
Project ImPACT. MSU research staff also will conduct twice-a-month cross-site video-based 
fidelity checks to ensure that Penn staff is providing training and feedback to the clinicians 
implementing the program in a way that is consistent with the Project ImPACT model.  
Clinician Fidelity will be measured using the Project ImPACT Coaching Fidelity Checklist.1 
Clinicians assigned to the two active treatment groups will self-monitor their coaching fidelity as 
part of the implementation procedure. We also will observe clinicians monthly to ensure accurate 
self-monitoring in the active treatment groups and to capture the extent to which clinicians in the 
treatment as usual group use any elements of ImPACT. Previous research indicates high interrater 
reliability between self and observer ratings on the coaching fidelity form (99% exact 
agreement);42 thus we anticipate that self-ratings will provide an accurate measure of clinician 
behavior. 
Cluster Randomized Study Design   
Six agencies will be blocked into groups of 3 based on agency size and location. Within each group 
of three, each agency will be randomized to one of three conditions: 1) treatment as usual; 2) 
ImPACT at a dose of 1 hour/week over 6 months; and 3) ImPACT at 4 hours/week over 6 months.   
Statistical Analysis for Aim 1 
Outcomes will be analyzed as continuous variables at the child or parent level across time. Initially 
all outcomes will be described using summary statistics (means, medians, and estimates of 
variance). The longitudinal nested random effects linear models will be used for each outcome; 
the models will consist of three levels of random effects to adjust for individual child/parent, 
agency and clinician variability. For each measure, the analysis dataset will contain one 
observation with each subject’s outcome measure at baseline and follow-up. The dependent 
variable in each model will be the measure of outcome at each time point. The independent 
variables will include a categorical variable for time period (baseline, 3 months, or 6 months), 
group (ImPACT_1hr/wk, ImPACT_4hr/wk or comparison), and their interaction. The magnitude 
and statistical significance of the beta coefficient for the interaction will provide an intent-to-treat 
assessment of the extent to which the intervention group improved relative to the comparison 

Project ImPACT Research Protocol 
 
Page 7 of 18 
Version 6 
group. Because agencies are randomly assigned, we expect clinicians and children in our 
intervention and comparison groups to be similar across baseline characteristics; however, if we 
find between-group differences we will control for them in our regression models. 
Aim 2: Examine the cost effectiveness of Project ImPACT at different doses. 
Overview 
The purpose of the cost evaluation is to determine both the cost of implementing Project ImPACT 
in Philadelphia, and the cost associated with any additional benefit of one treatment arm over the 
other. Both questions are of great interest to our community partners and system administrators in 
general. Our cost-effectiveness analysis will take two perspectives: 1) a service perspective, which 
is of particular interest to policy makers and includes hospital, community and school-based health, 
social and education services; and 2) a societal perspective to capture the full economic 
implications, which also include parental out-of-pocket costs (insurance copayments, 
unreimbursed therapies and materials etc.), child care and schooling costs, productivity losses 
(time off work due to child’s condition), and unpaid care. 
Measures 
Data on service and other resource use will be collected through parent report. To measure 
resources related to intervention implementation, we will document intervention implementation 
activities of agency partners, trainers/consultants, and experts from the investigative team in order 
to estimate total effort involved in implementation. Costs for the Project ImPACT groups will 
include: 1) initial training time of the agency clinicians; 2) subsequent consultation and coaching 
time provided to the clinicians; 3) clinician implementation of Project ImPACT with eligible 
children; and 4) consultant time necessary to maintain relationships with participating 
agencies/clinicians and support ongoing implementation of Project ImPACT. For each component, 
we will calculate two main types of costs: cost of physical materials used for training and 
interventions and costs associated with time spent for training, supervision, and intervention 
implementation. In addition, costs for initial training for the agency partners and 
trainers/consultants will be differentiated from the ongoing costs of implementation to provide 
separate estimates of costs for start-up in each agency and costs for ongoing implementation (with 
previously 
trained 
agency 
clinicians 
and 
trainers/consultants). 
Intervention 
and 
training/consultation times will be derived from the total Project ImPACT sessions delivered to 
each family each week and session logs completed by interventionists, trainers/consultants, and 
experts throughout the trial). To derive costs, we will use the resource costing method, which 
involves determining a price weight for each resource unit consumed where the price weight 
reflects the real opportunity cost of the service.47 This price weight is then multiplied by the 
number of units at that price and then summed over all services. We will estimate the costs of 
clinician, trainer, and expert supervisor time using average salary and benefits for those in job 
categories matching the staff providing these services. We will cost materials using acquisition 
costs. We will determine unit costs for Project ImPACT services from a modified version of the 
Drug Abuse Treatment Cost Analysis Program (DATCAP).48,49 This instrument is administered 
once and is used to derive price weights. This information includes salaries of personnel delivering 
services and other variable costs along with facility costs and other fixed costs associated with the 
intervention. The cost per hour will include provider costs and overheads (capital, administrative 
managerial) and travel costs. We will obtain other unit costs from published resources, national 
surveys, and government departments. We will calculate productivity losses using the human 

Project ImPACT Research Protocol 
Page 8 of 18 
Version 6 
capital approach, which involves multiplying time off work by the parent’s wage rate.50 We will 
calculate informal care costs using the market price approach that applies the price that would be 
paid if the care were provided by a formal care giver.51 Finally, we will calculate total costs for 
each agency and then averaged across centers within each study arm to determine average costs 
per study condition. 
Data Analysis 
The outcomes of different study arms will be combined with their respective costs to provide a 
measure of relative cost-effectiveness that can be compared to other interventions employing the 
same measure of outcome. The effects will be improvements in child and parent outcomes. Using 
the estimated mean cost and mean effect per patient by intervention group, we will construct an 
incremental cost-effectiveness ratio for each outcome for which Project ImPACT shows a 
significant effect. Incremental cost-effectiveness will be computed as the ratio of the difference in 
mean costs (incremental cost) to the difference in mean effects (incremental effect), and will 
represent the additional cost per additional improvement in child development and social 
communication scores and parent stress and self-efficacy scores, of one intervention arm compared 
to another. A joint distribution of incremental mean costs and effects for the intervention arms 
being compared will be generated using non-parametric bootstrapping.52 We will use these data to 
explore the probability that each treatment is optimal choice, subject to a range of possible 
maximum values (ceiling ratio) that a decision maker might be willing to pay for an additional 
improvement in child or parent outcome score. Cost-effectiveness acceptability curves (CEAC) 
will be generated by plotting these probabilities for a range of possible values of ceiling ratios.53,54 
CEACs are a standard decision-making approach to dealing with the uncertainty around the 
estimates of expected costs and effects and uncertainty regarding the maximum CER a decision 
maker would consider acceptable.54 We will examine baseline characteristics across intervention 
groups to determine whether randomization was successful in balancing observed baseline 
covariates. If there is unbalanced allocation in baseline characteristics, we will use net-benefit 
regression to estimate directly in a regression framework.
Statistical Power 
We base our statistical power analysis upon Aim 1, our primary aim, which tests the effects of 
Project ImPACT on child and parent outcomes. Prior studies of Project ImPACT and related 
parent-mediated interventions indi-cate a large effect on parent treatment adherence (.90-1.30), 
and a moderate effect on child outcomes (.34-.70).16,19,42 Meta-analytic reviews of studies of 
parent-implemented interventions for children with ASD report effects of .8 to 1.1.26 We used 
Power and Precision software, and assumed an alpha of 0.05, and that all tests are 2-tailed. We 
will have enrolled 3 clinicians in each agency and 3 children per clinician (18 subjects per inter-
vention group). Due to statistical software limitations in addressing multiple levels of clustering 
in power estima-tion, we adopted two perspectives. First, we explored potential effect sizes 
(Cohen’s d) with nesting of clinicians within agencies. Second, we estimated potential effect 
sizes with nesting of patients within clinicians. We as-sume an intra-class correlation of 0.05. In 
the first case, we calculate that we will have power of 80% to detect a moderate intervention 
effect of d=0.66. In the second case, we calculate that we will have power of 80% to de-tect a 
moderate intervention effect of d=0.53. 
PROTECTION OF HUMAN SUBJECTS 
Research will not begin until the Institutional Review Boards of the University of Pennsylvania 
and the City of Philadelphia approve the proposed study. 
Characteristics of Population 

Project ImPACT Research Protocol 
Page 9 of 18 
Version 6 
We anticipate the sample of parent-child dyads will be 45% African American, 26% Latino, 18% 
Caucasian, 6% Asian, 2% American Indian/Native American, and 3% multi-racial. Children will 
be less than 3 years  of age, and will either have a diagnosis of autism spectrum disorder or be 
identified as at-risk for autism by the early intervention system, as children under three years of 
age do not need a formal diagnosis to receive autism-specific treatment. We anticipate that 75% 
of the children will be male based on the sex distribution of autism. Participants will be excluded 
if they do not speak English. This is due to the procedures of the project, which include coding of 
videos of early intervention sessions. The team currently does not have capacity to code videos 
that are not in English. Participants will not be excluded based on their gender or past 
experience. 
Data to be collected from Human Subjects 
Data will take the form of demographic questionnaires, questionnaires, direct observations, direct 
assessments and video recordings. Data will be obtained directly from participants in the 
proposed research project.  
Field Observations: We will collect direct observation ratings of provider’s coaching fidelity 
using the Project ImPACT Coaching Fidelity Checklist. We will also collect direct observations 
of parent/child interactions that will inform the parent’s adherence to treatment using the Project 
ImPACT Intervention Fidelity Checklist and parent responsiveness using the Parenting 
Interactions with Children: Checklist of Observations Linked to Outcomes (PICCOLO). 
Demographic Questionnaires: We will collect sociodemographic information for each participant 
including information related to race/ethnicity, socioeconomic status, and education level.   
Questionnaires: We will collect questionnaires from parents examining their self-efficacy using 
the Early Intervention Parenting Self-Efficacy Scale (EIPSES), and on their quality of life using 
the World Health Organization QOL Assessment-Brief (WHOQOL-BREF).We will measure 
changes in social communication by administering the Vineland Adaptive Behavior Scales–3rd 
Edition with parents. We’ll also ask parents to fill out the Project ImPACT Parent Satisfaction 
Survey to evaluate how satisfied they were with the program overall.  
Assessments:  We will use the BOSCC to assess children’s social communication and play skills 
from self-recorded 10-minute parent-child play interactions.    
COVID-19 changes: All surveys will be administered electronically using a unique secure 
Redcap survey link. All sessions will be delivered via tele-intervention, consistent with current 
practice guidelines in the county. Fidelity observations will be conducted via tele-intervention.  
Service Use: We will gather data on service use by interviewing parents regarding the total and 
type of services their child receives. We will interview parents about service use at screening and 
update during the 3-month and 6-month timepoints using the Service Utilization Survey that the 
research team will fill out.  
COVID-19 changes: Interviews regarding service use will be conducted via telephone/video 
calls instead of in-person. 

Project ImPACT Research Protocol 
 
Page 10 of 18 
Version 6 
Recruitment and Informed Consent 
Recruitment 
Clinicians 
Clinician recruitment will occur through the directors of the participating agencies. Recruitment 
and retention strategies are based on our prior community trials in which we recruited 80-95% of 
eligible practitioners and experienced 5-7% attrition. Each agency director will distribute flyers to 
clinicians in their employ to describe the project and invite them to participate. Agency leaders 
also will describe the study at staff meetings. Team members will attend professional development 
and informational meetings throughout the winter and spring of 2020 to answer clinicians’ 
questions, address their concerns, and garner support. Study training activities will fulfill 
Pennsylvania’s annual training requirements for early intervention clinicians, which is a 
considerable economic benefit to the clinician and the agency. Each clinician will also receive up 
to $150  for completing the Project ImPACT training workshop. There will be a local number and 
e-mail address for staff to contact with any study-related questions. 
COVID-19 changes: Clinicians will be paid $150 for completing the training. The training is now 
being delivered across 4- 3.5 hour sessions, instead of 2 full days. Clinicians are not able to work 
during the training and are not paid by the county for the training time, as it is an external training. 
The compensation was increased to account for the increased time commitment across extended 
days for the remote training and subsequent loss of wages. 
Families 
Children and their families will be recruited and retained through clinicians and service 
coordinators who interact with families during evaluations and individualized family service 
planning meetings, with a flyer and brief video describing the study. As clinicians are recruited, 
they will refer families on their caseloads who have not yet responded. Participating families who 
successfully complete the measures will receive $50 at each data collection point. There will be a 
local number for families to call with any questions related to the study protocol. 
 
Consent Process 
All primary data collection will be conducted based on informed parental or practitioner consent. 
All signed consent forms will be maintained in the case file along with all identifying information 
in a locked cabinet that is kept in a secure office. The consent form includes the following 
elements: Purpose, Selection, Procedures, Risks, Costs and Financial Risks, Alternatives, Benefits, 
Payments, Confidentiality, and Withdrawal. 
 
Clinicians will be informed about the study via a letter from their agency director. The letter will 
describe the project and ask them to participate. Agency Directors will also describe the study and 
answer questions at staff meetings and distribute study flyers. Study team members will also attend 
professional development and informational meetings to address questions and concerns and 
garner interest and support. Clinicians who are interested will be able to reach out to the study 
team directly in order to start study procedures. 
 

Project ImPACT Research Protocol 
 
Page 11 of 18 
Version 6 
All six participating agencies will be randomized into three groups based on their agency size and 
location. Interested clinicians at each agency will be informed of the randomization group during 
the consent process and are free to decline participation at any time. 
 
Children and families will be recruited and retained through their clinician who is participating in 
the study as well as service coordinators at participating agencies who interact with families on a 
regular basis. Clinicians will refer families on their caseload and provide them with a flyer and 
brief video describing the study. Families who are interested in participating will reach out to a 
member of the study team in order to have questions and concerns addressed and see if they are 
eligible to participate. 
 
All participants will be required to sign a written consent form in order to participate. During this 
process, participants will have the opportunity to ask questions about these consent forms prior to 
giving written informed consent. This document includes a description of the entire study and 
expected roles and responsibilities of the practitioner, primary caregiver and child, risks and 
benefits to participants, and confidentiality procedures. The consent document will also clearly 
state that participation in the study is completely voluntary, and participants can drop out of the 
study without penalty at any time. Study staff will review this consent form at the first face-to-face 
contact. At this time, study staff will answer questions and obtain written consent from the 
participant. The consent forms explicitly outline the voluntary nature of participation in this 
research study. Easy channels of communication to study staff in the event that a participant wishes 
not to participate will be clearly outlined. Specifically, the contact information for the PI and study 
coordinator will be listed on the consent form, expectations for participation, compensation and 
further outlines of confidentiality practices specific to information collected. All consent 
documents follow the guidelines outlined by the Office of Human Subjects Protection of the 
National Institutes of Health, the University of Pennsylvania, the City of Philadelphia EI system, 
and ethical guidelines set forth by the state and federal governments. Documentation of subject 
consent will be stored in a locked file in the PI or designees locked office. 
 
Note: Per protocol changes, participants who signed earlier versions of the consent form that did 
not contain allowances for using videos for education purposes (consultation meetings) will be 
reconsented.  
 
In response to COVID-19 informed consent will be obtained electronically. First, an electronic 
enrollment packet will be e-mailed to individuals who have expressed interest in survey 
completion. The enrollment packet will include an electronic, informed consent form via the 
University of Pennsylvania’s Qualtrics portal, and an informational flyer about the survey 
component of the study. The research team will provide detailed information about the study via 
phone, and participants will complete a screening eligibility survey. If they meet eligibility criteria 
as outlined in the research protocol, they will view the informed consent document. The consent 
document includes a description of the study and expected roles and responsibilities of the 
provider, risks and benefits to participants, and confidentiality procedures. The consent document 
will also clearly state that participation in the study is completely voluntary, and participants can 
drop out of the study without penalty at any time. Consent will be obtained through the Qualtrics 
electronic survey consent form 

Project ImPACT Research Protocol 
 
Page 12 of 18 
Version 6 
 
Subject Compensation 
 
Participating providers and families will be compensated for time spent completing data collection 
steps. 
 
Providers 
Clinicians in the Treatment as usual group will receive $25 for completing self report fidelity 
measures after each video recorded observation of their early intervention session. We will observe 
3 sessions per family and enroll up to 3 families per provider. They may receive up to $225 for 
completed self-report fidelity after observations (3 observations at $25 each x 3 families) 
 
Clinicians enrolled in one of the ImPACT groups and attend the 2 day project ImPACT interactive 
workshop and complete the self-paced, online training course, will be compensated $ 150 for their 
time. They will also receive the same compensation schedule as clinicians in the Treatment as 
Usual group for the self report fidelity measures.  
 
Families 
Families will receive $50 at each data collection point. There will be three data collection points 
for a total of up to $150. 
 
Data Sharing 
 
If providers believe that their performance in the study may be evaluated by their agency or the 
EI system they may become anxious and thus make them unlikely to participate in the study. 
Videotaping provides concrete examples of coaching techniques that enable efficient and 
accurate measurement of the strategies used during intervention sessions; however, there is 
potential that such videotaping will provide concrete examples of intervention techniques or 
other behavior that could negatively influence staff performance review if administrators or 
supervisors in the EI system were to view it. For this reason, there will be no disclosure of 
videotaped performance or self-reported information about provider fidelity, especially to these 
personnel. As a result, the research team will keep all identifiable and individual participant data 
confidential from all administrative personnel, including agency executive and clinical directors, 
and any other staff member of the Department of Behavioral Health and Intellectual Disabilities.   
Written notification of this procedure will be provided to all involved parties at the Philadelphia 
Department of Public Health. However, upon conclusion of data analysis, the research team will 
provide anonymous and aggregate results of the study to the Philadelphia Department of Public 
Health, the department of Intellectual and Disability Services within DPH, as well as any 
participating community agencies. Anonymous and aggregate results provided to community 
agencies will be aggregate data from all participants, not exclusively those from any single 
community agency. This procedure acts in the interest of participant confidentiality and job 
security. 

Project ImPACT Research Protocol 
 
Page 13 of 18 
Version 6 
Potential Risks to Subjects 
 
Risk to children and parents.  
Parent and child involvement in the proposed study involves direct observation measures of 
parents and their children in their homes and the parents’ completion of measures about their 
children’s behavior and their own self-efficacy. Additionally, all children will be assessed to gather 
information about their social-communication and language skills. The study provides training to 
parents and to the clinicians working with their children that otherwise would not be available. 
Parents’ time commitment across all study arms is similar to usual practice. The study therefore 
involves minimal risk to children and their families. The risks include:  
 
1) Respondent burden associated with the time and effort to complete the proposed battery;  
a. We estimate time burden to be an average of 1 hours for data collection from the 
child at baseline and at 6 months. Time burden for parents will average 85 minutes 
at baseline, 3 and 6 months. Relative to other studies that involve clinical 
assessment of children with ASD, this is a relatively small time commitment. 
2) The possibility of breached confidentiality by disclosure of personal information; 
3) The risk of distress caused by participating in the intervention;  
a. The proposed measures include a number of sensitive items related to children’s 
clinical presentation and family characteristics. Evaluations of children may cause 
them fatigue or anxiety. Some questions asked of parents (such as those about 
income) may cause parents some discomfort. Risks involved with participating in 
the child assessments are expected to be minimal, given that process will be 
supervised by a trained clinical psychologist. 
b. Parents may feel anxious by observations or videotaping, especially if they believe 
that the research team is evaluating their performance. Videotaping provides 
concrete examples of teaching techniques that are vital to the measurements 
proposed in this research. However, there is potential that such videotaping will 
provide examples of behavior that the research team believes endanger the parent 
or child. For this reason, there will be no disclosure of observed or videotaped 
information to those outside of the research team unless there is a suspicion of child 
abuse or neglect. Parents will be informed during the consent process that all study 
personnel are mandated reporters. All staff are trained in mandated reporting and 
we have an approved reporting protocol for all of our community work. In cases of 
suspected abuse or neglect, the research staff member suspecting the abuse or 
neglect will contact one of the two licensed psychologists on the study. They will 
review the situation together and determine if it meets criteria for reporting. 
4) The risk that the intervention may have some iatrogenic effect. 
a. There is a small chance that the proposed intervention may have negative 
consequences. The majority of evidence suggests that the components of the 
proposed intervention are the most efficacious in educating children with ASD; 
their effectiveness in this particular setting has not been tested, however. 
 
Risk to clinicians 

Project ImPACT Research Protocol 
 
Page 14 of 18 
Version 6 
Clinician participation in the proposed study involves direct observation measures of their 
implementation of their practice. It also involves completing measures on their practice. The study 
therefore involves minimal risk to clinicians. Possible risks include:  
1) Respondent burden associated with the time and effort to complete the proposed battery; 
a. We estimate that the time burden to clinicians for the proposed measures is 65 
minutes at baseline, 3- and 6-month time points. Observations of clinician’s 
practice will occur during their regularly scheduled home visits. Completing the 
fidelity checklist will take 5 minutes per visit.  
2) Distress caused by the questionnaires, the interview or fidelity measures;   
a. Providers may be made anxious by videoing, especially if they believe that their 
performance in the study may be evaluated by their agency or the EI system. 
Videotaping provides concrete examples of coaching techniques that are vital to 
the measurements proposed in this research; however, there is potential that such 
videotaping will provide concrete examples of intervention techniques or other 
behavior that could negatively influence staff performance review if 
administrators or supervisors in the EI system were to view it. For this reason, 
there will be no disclosure of videotaped performance or self-reported information 
about provider fidelity, especially to these personnel. To help minimize this risk, 
we will provide EI administrative personnel, including agency executive and 
clinical directors, written notification that these measures are confidential and at 
no time and under no circumstances will be shared with EI personnel.  
b. Potentially more distressing are emotional responses to fidelity measures. The 
fidelity measures are designed to quantify EI providers’ adherence to the 
intervention protocol. EI providers may feel as though their willingness or ability 
to treat children with ASD is being challenged. Specifically, documenting 
providers’ performance may cause feelings of insecurity regarding effort or ability 
 
3) The possibility of breached confidentiality by disclosure of personal information to agency 
or EI system personnel linking individual clinicians to their performance.  
 
 
Protection against Risks 
 
Interaction with research participants. To minimize distress associated with respondent burden and 
distress associated with the proposed measurement batteries, both parents and clinicians will be 
told at the beginning of each observation, or have stated at the beginning of each questionnaire 
that they may refuse to participate in any part of the study and skip any question at any time during 
the interview. We will accommodate parents’ wishes with regards to the timing of the data 
collection and assessment, will break up the data collection into stages if requested, and will hire 
and train research staff to be sensitive to the well-being of children, parents and clinicians. 
Participants will be reminded that information will be kept confidential and will never be published 
in any manner that would identify them. Children’s distress is most likely to arise from stress or 
fatigue associated with testing, frequent breaks will be offered along with snacks if desired. Tests 

Project ImPACT Research Protocol 
 
Page 15 of 18 
Version 6 
will be administered to children by trained professionals experienced in the procedures and 
sensitive to participants’ clinical conditions. All participating families will be reminded that all 
clinical data will be collected for research purposes only. Families will be encouraged to contact 
their pediatrician, school evaluation center, or local developmental specialists if they desire further 
evaluation. A list of community referral resources will also be provided to all families. 
Data Management and Access. To minimize risks associated with breaches of confidentiality, all 
data on study participants will remain confidential at all times. The exposure of the identity of 
study participants will be avoided wherever possible. An 11 digit identification number will be 
employed to encode the participant identity on study materials, including data collection forms. 
The Center for Mental Health at Penn, where data will be stored, has a 20-year history of 
conducting research using protected health information and linking it with other datasets. Center 
protocols for data management and protection have been developed and implemented for all 
datasets that include protected health information. The research team will handle the original 
media on which data are kept. Media will be labeled confidential, use only unique identification 
numbers and be kept in a locked filing cabinet. Penn’s technology support services employ a 
number of security and privacy technologies and best practices in order to deliver services that 
provide “security in depth”. 
To ensure the privacy and confidentiality of data for this project we will only store and use 
the identifiable data at the following locations: 1) Password-protected PCs in locked offices at 
Penn. 2) A server at the University of Pennsylvania within the facilities managed room in the data 
center. A Senior LAN Consultant will upload the data onto the secure production servers. Videos 
will be sent from Penn to Dr. Ingersoll’s lab at Michigan State University for coding. Videos will 
be sent via Penn Box, Penn’s secure file sharing service. When viewing videos at MSU, staff will 
keep the video copy in Penn Box and not make copies to other media.  Lastly, all output containing 
individual identifiable information is treated as confidential data. This information is never 
transferred electronically via email or other protocols. Any printed material containing individual 
identifiers is shredded.  
Physical Security – Servers are hosted in dedicated virtual machines running on Dell SAN 
hardware in the Information Systems and computing facilities managed computer room. The 
hosting environment SAN and Host Bus Adapters are in a locked Dell rack housed in the Computer 
Room with restricted door card access to authorized personnel only. There is 24-hour camera 
surveillance. 
Business Continuity – Server environments get redundant power from independent power feeds. 
In addition, each of the power sources is UPS protected. There is a Halon fire suppression system, 
with alarm points below the raised floor and in the ceiling. The virtual machines are hosted in 
VMWare ESX environments connected to a Dell SAN for shared storage. All virtual machines are 
stored on the SAN. To minimize downtime caused by hardware failure, the environment is 
configured for redundancy with multiple ESX hosts, multiple Host Bus Adapters, multiple SAN 
Switch ports, multiple storage processors, and RAID 5 storage. The server environments are 
replicated to a secondary site at 3650 Chestnut Street, which has equivalent physical security and 
also employs all of the security best practices outlined below. 
Server Administration and Maintenance – The servers are administered by a team of four full-time 
professional IT staff. The team has over 50 years of combined experience configuring and 

Project ImPACT Research Protocol 
 
Page 16 of 18 
Version 6 
supporting Windows servers, and all of the staff attend technical training regularly to stay current 
in best practices for configuring and securing Windows and VMWare environments. 
Server Security Best Practices – In keeping with SANS and Microsoft best practices, all software 
services and corresponding ports on the servers that are known to be substantial security risks and 
which are not used by CMH resources are disabled including telnet, and ftp. Security patches are 
applied promptly and there are standard processes in place for preventive maintenance and 
monitoring of the servers. 
Hardware Firewalls - All servers reside behind a cluster of Juniper SSG520 firewalls. A Juniper 
SSG520 appliance configured in high availability mode acts as a secure gateway between 
PennNet/Internet (Untrusted Zone) and server resources behind the firewall. A custom firewall 
policy is developed for each resource hosted on each of the virtual machines. All policies are 
developed based on service port and PennNet only IP addresses.  
VLAN - A Foundry Switch in the TSS managed rack is configured for 4 VLANs to support traffic 
segmentation behind the SSG520. LTS has access to each VLAN, giving us the ability to place 
each virtual machine in the appropriate VLAN for another layer of security. 
SSL Encryption for Web Services - When hosting web servers we only support the use of HTTPS 
with an SSL certificate to minimize vulnerabilities and exploits common on the standard HTTP 
port. All port 80 traffic is re-directed to port 443. 
Password Policy - A complex password policy, meeting Microsoft complex password 
requirements, is in effect in the CMHPSR domain. The password policy requires all CMHPSR 
domain users to change their domain password every 180 days. 
Preparation of Staff for Data Collection 
All study personnel will have Patient Oriented Research certification from the University of 
Pennsylvania. Clinical personnel meeting with families and providers will hold a Bachelor’s, 
Master’s or doctorate degree. The PI will oversee the additional training and ongoing review of 
interviewers. Clinical research staff who conduct assessments will be trained to reliability and 
supervised by the clinical psychologist on staff. The PI will also provide all research assistants 
with training regarding culturally sensitive practices, and maintaining professional boundaries 
while visiting families’ homes. The research team will travel in pairs for home-based visits as a 
safety measure for both participants and research team members.   
To prepare Penn staff to consult in the field, Dr. Ingersoll and the MSU research team will train 
them to fidelity in Project ImPACT and its coaching model, and fidelity monitoring. This training 
will include the same introductory training community clinicians will receive as described in Aim 
1, consultation meetings with MSU study staff to provide feedback on program implementation 
with a family with ASD via videoconferencing; and obtaining reliability on parent treatment 
adherence and coaching fidelity scoring for master session tapes. Once Penn staff have been 
trained to fidelity in coaching and reliability in fidelity monitoring, they will meet biweekly with 
MSU staff via video-conferencing for supervision as they provide monthly consultation sessions 
to community clinicians using Project ImPACT. MSU research staff also will conduct twice-a-

Project ImPACT Research Protocol 
Page 17 of 18 
Version 6 
month cross-site video-based fidelity checks to ensure that Penn staff is providing training and 
feedback to the clinicians implementing the program in a way that is consistent with the Project 
ImPACT model.  
Adverse Events 
All unexpected adverse events will be reported by study personnel to the DPH IRB, the Project 
Officer at IES, and the IRB at Penn via standard adverse event reporting procedures. All adverse 
reactions will be noted and discussed thoroughly with the PI, mentors, and IRB to determine proper 
reporting procedures on a case-by-case basis. 
Potential Benefits of the Proposed Research to the Participants and Others  
There are some benefits to family/child and clinician participants. Families in the treatment 
conditions will receive training in intervention techniques considered best practice for young 
children with ASD. In addition, clinicians in the two treatment conditions will receive considerable 
training and support throughout their participation that has the potential to benefit the children they 
treat. The potential exists for great benefits to child-serving systems at large. Potential risks are 
reasonable given the safeguards proposed and the valuable information to be yielded regarding 
interventions for children with ASD. 
Data and Safety Monitoring Plan 
Since this study constitutes a clinical trial, although we are not studying experimental drugs, the 
intervention is experimental with the risks described in the Human Subjects section. Therefore, it 
is appropriate to have a Data and Safety Monitoring Board (DSMB) to monitor the study with an 
attendant monitoring plan as outlined below. The role of the DSMB will be to ensure that no 
unexpected untoward consequences of the intervention occur, and if they do, to implement 
safeguards to decrease or eliminate future risks. Any adverse events will be provided to the DSMB 
in real time. These adverse events include, but are not limited to, self-injurious behavior or 
inappropriate or threatening behavior with parents, or clinicians or research staff. Less serious 
adverse events related to the intervention will be compiled and summarized for the DSMB at their 
scheduled meetings. All other severe adverse events will be monitored by the principal 
investigator, the study coordinator, and the assessment team and will be reported to the DSMB 
regardless of a priori expectation or investigator assessment of likelihood of the study being the 
cause of the adverse event. The DSMB also will have the power to request further data from the 
principal investigator based on any concerns raised in the ongoing investigation. At any time 
during the study, based on its findings, the DSMB can put the study on either temporary hold or 
stop the study entirely after notifying the principal investigator and the IRB. In the rare case in 
which they must do so, the DSMB will generate a formal report to the principal investigator, the 
sponsor, and the IRB. This report would then be modified for a lay reader, a version for past 
participants and current participants would be produced, and the letter would be sent to all past 
and current study participants. The DSMB will first meet at the beginning of the study to review 
its charge and activities. It will then meet quarterly to review non-serious and serious adverse event 
rates, as well as preliminary efficacy data. It also will meet on an ad hoc basis if the real-time 
serious adverse events warrant meeting, in their judgment. The format of the meeting will include 
a presentation of the summarized data by the Principal Investigator (Pellecchia) and the 

Project ImPACT Research Protocol 
Page 18 of 18 
Version 6 
biostatistician (Cidav). The absolute rates of adverse events as well as the rate compared between 
the intervention and control groups will be presented. The Board will be composed of two faculty 
at Penn who are not associated with this intervention but have familiarity with autism intervention, 
and a senior staff member from Philadelphia’s early intervention program.