ECHO Autism Replication: Step Wedge Randomized Trial
Phase: NA
Conditions: Autism Spectrum Disorder
Sponsor: Massachusetts General Hospital

Autism Intervention Research  
CONFIDENTIAL 
Page 1 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
 
ECHO AUTISM  
 
Study Chair: 
 
Kristin Sohl, MD 
Medical Director 
Thompson Center for Autism and other Neurodevelopmental Disorders 
 University of Missouri 
 
Co-PI(s): 
 
Evdokia Anagnostou, MD 
University of Toronto 
 
Amanda Bennett, MD, MPH 
Children’s Hospital of Philadelphia  
 
Kristin Dalope, MD, MEd 
University of Pittsburgh  
 
Susan Hyman, MD 
University of Rochester 
 
Heather Johnson, PysD 
Cincinnati Children’s Hospital Medical Center 
 
Maya Lopez, MD 
University of Arkansas for Medical Sciences 
 
Micah Mazurek, PhD 
University of Missouri 
 
Ann Neumeyer, MD 
Lurie Center for Autism  
 
 
 
 
 
 
 
 
Karen Ratliff-Schaub, MD 
Nationwide Children’s Hospital  
 
Beth Malow, MD, MS 
Vanderbilt University Medical Center 
 
Robin Steinberg-Epstein, MD 
University of California, Irvine 
 
Supported by: 
 
Heath Resources and Services Administration 
 
Grant Number: UA3MC11054 

Autism Intervention Research  
CONFIDENTIAL 
Page 2 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
TABLE OF CONTENTS 
1. 
ETHICS/PROTECTION OF HUMAN SUBJECTS ................................................................................................. 9 
1.1 
INSTITUTIONAL REVIEW BOARD (IRB) ...................................................................................................................... 9 
1.2 
ETHICAL CONDUCT OF STUDY ................................................................................................................................. 9 
1.3 
SUBJECT INFORMATION AND CONSENT ..................................................................................................................... 9 
1.4 
SUBJECT INCLUSION .............................................................................................................................................. 9 
1.5 
STUDY MODIFICATION/DISCONTINUATION .............................................................................................................. 10 
2. 
BACKGROUND ............................................................................................................................................ 10 
2.1 
RATIONALE ....................................................................................................................................................... 10 
2.2 
SUPPORTING DATA ............................................................................................................................................. 11 
2.3 
RISKS AND BENEFITS ........................................................................................................................................... 11 
2.3.1 
Risks ..................................................................................................................................................... 11 
2.3.2 
Benefits ................................................................................................................................................ 11 
3. 
OBJECTIVES ................................................................................................................................................. 12 
3.1 
STUDY OBJECTIVES: TO DETERMINE WHETHER PARTICIPATION IN A COLLABORATIVE TELEHEALTH INTERVENTION WILL RESULT IN 
IMPROVED LEARNING, CLINICAL PRACTICE BEHAVIOR AND EFFICACY AMONG PRIMARY CARE PROVIDERS (PCPS). ............................. 12 
3.2 
STUDY OUTCOME MEASURES ............................................................................................................................... 12 
3.2.1 
Primary outcome measures ................................................................................................................. 12 
3.2.2 
Secondary measures ............................................................................................................................ 12 
4. 
STUDY DESIGN ............................................................................................................................................ 13 
4.1 
OVERALL STUDY DESIGN AND PLAN ....................................................................................................................... 13 
4.2 
STUDY CENTERS ................................................................................................................................................. 14 
4.3 
STUDY ENROLLMENT PROCEDURES ........................................................................................................................ 14 
4.3.1 
Recruitment of Participants ................................................................................................................. 14 
4.3.2 
Consent and Assent .............................................................................................................................. 15 
4.4 
STUDY DURATION............................................................................................................................................... 15 
4.5 
PARTICIPANT REMUNERATION .............................................................................................................................. 15 
4.6 
PROTOCOL ADHERENCE ....................................................................................................................................... 15 
5. 
STUDY ENROLLMENT AND WITHDRAWAL .................................................................................................. 15 
5.1 
NUMBER OF STUDY SUBJECTS ............................................................................................................................... 15 
5.2 
INCLUSION AND EXCLUSION CRITERIA ..................................................................................................................... 16 
5.2.1 
Subject Inclusion Criteria ...................................................................................................................... 16 
5.2.2 
Subject exclusion criteria ..................................................................................................................... 16 
5.3 
TREATMENT ASSIGNMENT PROCEDURES ................................................................................................................. 16 
5.3.1 
Randomization procedures .................................................................................................................. 16 
5.3.2 
Reasons for withdrawal ....................................................................................................................... 16 
5.3.3 
Handling of withdrawals ...................................................................................................................... 16 
6. 
STUDY INTERVENTIONS .............................................................................................................................. 16 
6.1 
INTERVENTIONS, ADMINISTRATION AND DURATION .................................................................................................. 16 
6.2 
ADHERENCE ASSESSMENT .................................................................................................................................... 17 
7. 
STUDY SCHEDULE ....................................................................................................................................... 17 
7.1 
SCREENING ....................................................................................................................................................... 17 
7.1.1 
Screen Failures ..................................................................................................................................... 17 
7.2 
ASSESSMENTS .................................................................................................................................................... 18 
7.2.1 
Provider-Completed Measures............................................................................................................. 18 
7.2.2 
Chart Review Measures ....................................................................................................................... 18 

Autism Intervention Research  
CONFIDENTIAL 
Page 3 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
7.2.3 
Process Measures ................................................................................................................................ 19 
7.3 
PROTOCOL DEVIATIONS ....................................................................................................................................... 19 
8. 
CLINICAL ASSESSMENTS AND OUTCOME MEASURES .................................................................................. 19 
8.1 
PRIMARY OUTCOME MEASURES ............................................................................................................................ 19 
8.2 
SECONDARY MEASURES ....................................................................................................................................... 20 
8.3 
INTERVENTION FIDELITY EVALUATIONS ................................................................................................................... 20 
9. 
STATISTICAL ANALYSIS PLAN ...................................................................................................................... 20 
9.1 
STATISTICAL CONSIDERATIONS .............................................................................................................................. 20 
9.1.1 
Data Analysis ....................................................................................................................................... 20 
9.1.2 
Power Considerations .......................................................................................................................... 22 
10. 
DATA COLLECTION, MANAGEMENT, AND MONITORING ............................................................................ 22 
10.1 
ROLE OF DATA MANAGEMENT ......................................................................................................................... 22 
10.1.1 
Web-Based Data Collection and Management System .................................................................. 22 
10.1.2 
Certification in the Use of Web-Based Data Entry System .............................................................. 22 
10.1.3 
Data Entry and Checks .................................................................................................................... 22 
10.1.4 
Quality Assurance ........................................................................................................................... 23 
10.2 
DATA HANDLING AND RECORD KEEPING ............................................................................................................ 23 
10.2.1 
Confidentiality ................................................................................................................................. 23 
10.2.2 
Retention of records ........................................................................................................................ 24 
10.2.3 
Publications ..................................................................................................................................... 24 
10.2.4 
Data Sharing ................................................................................................................................... 24 
11. 
REFERENCES ............................................................................................................................................... 25 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 4 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
PROTOCOL SYNOPSIS 
 
Study Title 
 
ECHO Autism   
 
Version Number 
 3 
 
Rationale for the Study 
 
ASD is associated with significant impairments in social, communication and behavioral 
domains and medical and psychiatric comorbidities [1-3]. Unmanaged comorbid conditions 
contribute to increased stress and burden for families [4-5]. Children with ASD have increased 
risk for unmet healthcare needs, which is exacerbated among underserved populations [6-8]. 
Individuals in rural areas face socioeconomic and geographic barriers to accessing health 
care, including high rates of poverty [9]. They are more likely to lack health Insurance [10]. 
Rural areas have significant shortages of specialists, necessitating long distance travel for 
families to access services [10]. Rural children have higher rates of unmet medical and dental 
needs and of emergency department visits than do non-rural populations [11].  
 
With increasing prevalence of ASD, diagnostic and treatment demands far exceed the 
capacity of specialty centers [12]. Early identification, referral and effective treatment are 
essential for enhancing outcomes, yet children with ASD face delays in diagnosis [13]. 
Although PCPs provide immediate and community-based care for children, they often feel ill-
equipped to care for children with ASD [14-15]. Children with ASD experience co-occurring 
conditions that PCPs could manage, most notably sleep problems and constipation [3, 16-17]. 
However, PCPs report lack of knowledge and confidence in treating children with ASD, with 
resulting unmet healthcare needs [14-15]. PCPs also may lack the knowledge to manage 
ongoing psychotropic medication use. Lack of PCP comfort caring for children with ASD likely 
contributes to higher rates and duration of hospitalizations, greater expenditures, and greater 
use of psychotropic medications in children with ASD [18-21]. Thus, there is a critical need to 
improve early identification of ASD by PCPs and to enhance PCP effectiveness in managing 
sleep, constipation, psychotropic medications and other co-morbidities for children with ASD. 
 
Study Design 
 
A cluster-randomized design will be used, with sequential, staggered roll-out of the ECHO 
Autism intervention to 5 clusters of participants over a 1 year period. This design was chosen 
to maximize our ability to determine effectiveness of the intervention while minimizing potential 
contamination across groups and addressing potential ethical concerns. First, it would be 
problematic to randomize at the level of individual participants due to potential contamination 
across groups. Second, the staggered roll-out allows for comparison of each cluster to 
contemporaneous control groups as well as to its own baseline. Lastly, there will be a large 
benefit from the intervention for the participants themselves and we expect a large benefit for 
the patient populations that they serve as well.  Moreover, for the actual participants (PCPs) 
there is no potential harm (as they receive CME credits for participation). Therefore, 
randomization of participants to never receiving the intervention would be inappropriate. To 
enable a rigorous assessment of the effect of the ECHO intervention, we are randomizing 
different centers (and the participants linked to each center) to starting at different times during 

Autism Intervention Research  
CONFIDENTIAL 
Page 5 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
the study and will be using multiple data collection points under baseline, ECHO and follow-up 
conditions.  
 
Study Objectives and Endpoints  
 
Primary Objective: To determine whether participation in a collaborative telehealth 
intervention will result in improved learning, clinical practice behavior and efficacy among 
primary care providers (PCPs).  
 
Hypothesis 1: Following participation in ECHO Autism, PCPs will demonstrate 
significant improvements in ASD knowledge as assessed by pre- to post-intervention 
knowledge tests in ASD screening and identification and assessment and treatment of 
medical co-morbidities;  
 
Hypothesis 2: Following participation in ECHO Autism, PCPs will demonstrate 
significant improvements in clinical practice/behavior as assessed by pre- to post-
intervention chart reviews in ASD screening (co-primary outcome) and treatment of 
medical co-morbidities, in particular, sleep problems and constipation (co-primary 
outcome). 
 
Hypothesis 3: Following participation in ECHO Autism, PCPs will demonstrate 
significant improvements in self-efficacy in ASD screening and identification and 
treatment of medical co-morbidities. 
 
Intervention and Duration 
 
As noted above, a cluster-randomized design will be used, with sequential, staggered roll-out 
of the ECHO Autism intervention to 5 clusters over a 1 year period.  The intervention will be 
delivered at 10 ECHO Autism Hubs, which will be randomly assigned to one of 5 clusters.  
Cluster assignment will determine the timing of the intervention start-date for each ECHO 
Autism Hub.  Each ECHO Autism Hub will deliver the intervention to 15 PCP participants (for a 
total of 150 participants). 
 
Each ECHO Autism Hub will be comprised of a team of up to 5 autism specialists 
(Physician/Autism Medical Specialist, Psychologist, Family Navigator, Dietician, and Parent 
Expert). The ECHO Autism Leadership team at the University of Missouri (MU) and the 
Replication Support Team at the University of New Mexico will train each ECHO Autism Hub in 
delivery and implementation of the intervention.  
 
During the intervention phase, each ECHO Autism Hub team will provide twice-monthly 2-hour 
ECHO Autism Clinics for 15 PCP participants during a 6-month period. Each Clinic will utilize 
high quality, secure video conferencing technology to allow PCPs to interface with the ECHO 
Autism Hub team and all other participants, view documents, and view videos on screen (with 
minimal technological requirements for participants). The intervention will follow the protocol 
previously developed and tested by the ECHO Autism Leadership team. Based on this 
protocol, each ECHO Autism Clinic will include a didactic presentation, 2 to 3 PCP-generated 
case presentations, expert feedback and group discussion. Although the ECHO Clinic will 
include discussion of specific cases, no identifiable personal health information will be shared, 
individual patients will not be identified, and no direct patient care will be provided (PCP 
participants will maintain responsibility for care of their patients, but will develop new clinical 
skills through guided practice and collaborative learning). ECHO Autism didactic presentations 

Autism Intervention Research  
CONFIDENTIAL 
Page 6 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
will include use of AIR-P toolkits, guidelines, and algorithms to enhance medical care of 
children with ASD, with particular emphasis on identification/screening and management of co-
morbid conditions. This combination of case-based learning, co-management with autism 
specialists, and didactic presentations provides multiple learning modalities for enhancing PCP 
knowledge and expertise. 
 
Study Locations 
 
The study will be implemented at 10 different ATN Sites (ECHO Autism Hubs):  15 PCP 
participants per ECHO Autism Hub will be recruited from the geographic region in which the 
ATN site is located. 
 
1. Children’s Hospital of Philadelphia 
2. Lurie Center for Autism 
3. University of Pittsburgh Medical Center 
4. University of Rochester 
5. Cincinnati Children’s Hospital Medical Center 
6. Nationwide Children’s Hospital 
7. Arkansas Children's Hospital/UAMS 
8. Vanderbilt University Medical Center 
9. The Center for Autism & Neurodevelopmental Disorders at UC Irvine 
10. Toronto ATN Site (Holland Bloorview Kids Rehab) 
 
Number of Planned Subjects 
 
15 PCP participants from each ECHO Autism Hub will be enrolled (total enrollment = 150 
participants)  
 
Study Population 
 
Participants will include primary care providers (PCPs) who provide care to children, whose 
patient populations are at least 50% underserved.  
 
Assessment Groups 
 
As noted above, a cluster-randomized design will be used, with sequential, staggered roll-out 
of the ECHO Autism intervention to 5 clusters over a 1 year period.  The intervention will be 
delivered at 10 ECHO Autism Hubs, two being randomly assigned to each of the 5 clusters.  
Cluster assignment will determine the timing of the intervention start-date for each ECHO 
Autism Hub.  Each ECHO Autism Hub will deliver the intervention to 15 PCP participants (for a 
total of 150 total participants).  Participants will be assessed at four time points (as described 
below). 
 
Duration of Assessment and Follow-up 
 
Each PCP participant will complete a battery of assessments at four time points:  
Baseline/Pre-Intervention (T1), Mid-Intervention (T2), Post-Intervention (T3), and Follow-up 
(T4).  The duration of the ECHO intervention will be 6 months, and the interval between each 

Autism Intervention Research  
CONFIDENTIAL 
Page 7 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
assessment point will be 3 months. The timeline of intervention and assessments by cohort is 
shown below: 
 
 
12/1/2016 
3 Months 
3/1/2017 
3 months 
6/1/2017 
3 months 
9/1/2017 
3 months 
12/1/2017 
3 months 
3/1/2018 
3 months 
6/1/2018 
3 months 
9/1/2018 
Cohort 1 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
 
 
 
 
 
 
 
Cohort 2 
 
 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
 
 
 
 
 
Cohort 3 
 
 
 
 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
 
 
 
Cohort 4 
 
 
 
 
 
 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
 
Cohort 5 
 
 
 
 
 
 
 
 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
Measures  
 
Primary outcome measures include: ASD knowledge, clinical practice/behavior and self-
efficacy. ASD knowledge will be assessed at all timepoints using a 33-item test developed 
specifically for this study. Clinical practice/behavior will be assessed at all timepoints by chart 
review of a subset of charts from each PCP’s practice. Self-efficacy in ASD screening and 
identification and treatment of medical co-morbidities will be assessed at all timepoints using a 
57-item questionnaire that was developed for an ECHO Autism pilot study.  
 
Secondary measures include: demographic and practice information, satisfaction, perceived 
barriers, participation, and a precise schedule of ECHO topics and dates of PCP chart 
reviews.  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 8 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
Glossary of Abbreviations 
 
AAP 
American Academy of Pediatrics  
AIR-P 
Autism Intervention Research Network on Physical Health 
ASD 
Autism Spectrum Disorder 
ATN 
Autism Treatment Network  
DCC 
Data Coordinating Center 
ECHO  
Extension for Community Healthcare Outcomes  
HIPAA  
Health Insurance Portability and Accountability Act 
HRSA 
Health Resources and Services Administration  
IRB 
Institutional Review Board  
OHRP 
Office for Human Research Protections  
PI 
Principal Investigator 
SID 
Study Identification Number  
UNM 
University of New Mexico  

Autism Intervention Research  
CONFIDENTIAL 
Page 9 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
1. 
ETHICS/PROTECTION OF HUMAN SUBJECTS  
 
1.1 
Institutional Review Board (IRB) 
 
This protocol and the recruitment letter (Appendix B) and any subsequent modifications will 
be reviewed and approved by the IRB or ethics committee responsible for oversight of the 
study. The recruitment letter will describe the purpose of the study, the procedures to be 
followed, and the risks and benefits of participation.  A copy of the letter will be given to the 
participant, and this fact will be documented in the participant’s record. 
 
1.2 
Ethical Conduct of Study  
 
This study will be conducted using good clinical practice (GCP), as delineated in Guidance 
for Industry: E6 Good Clinical Practice Consolidated Guidance, and according to the criteria 
specified in this study protocol.  Before study initiation, the protocol and the informed 
consent documents will be reviewed and approved by an appropriate IRB/REB.  Any 
amendments to the protocol or to the consent materials must also be approved by the AIR-P 
CCC, AIR-P DCC, and appropriate IRB before they are implemented. 
 
Compliance with 42 CFR Part 93, Public Health Service (PHS) Policies on Scientific 
Misconduct is implicit in the application for this proposal.  The academic institutions 
participating in the ATN and this proposal have approved assurances and required renewals 
on file with the Office of Research Integrity (ORI) and compliance with these policies and 
procedures and the requirements of part 93 are in place.  We understand and abide by the 
definitions of research misconduct per PHS policies (fabrication, falsification, or plagiarism in 
proposing, performing, or reviewing research, or in reporting research results). 
 
1.3 
Subject Information and Consent  
 
Participants will include 150 primary care providers.  Inclusion/exclusion, recruitment, 
consent, and enrollment are described in the sections below. 
 
1.4 
Subject Inclusion 
 
Inclusion criteria for PCP participants are as follows: 
 
Current practice as a primary care provider (PCP).  
 
Currently providing care for children.   
 
Professional training in:  general pediatrics, family medicine, advanced practice nursing 
(i.e. nurse practitioner or physician assistant).   
 
Active medical license in the state of practice. 
 
Patient population is at least 50% underserved. 
 
Exclusion criteria are as follows: 
 
Not currently practicing as a primary care provider. 
 
Not currently providing care for children. 
 
Trainee status (e.g., medical student, intern, resident, or other pre-professional trainee). 
 
Subspecialist (e.g., psychiatrists, neurologists, developmental and behavioral 
pediatricians). 

Autism Intervention Research  
CONFIDENTIAL 
Page 10 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
 
Practicing within the same practice as another PCP Participant (i.e., only one PCP 
participant from any given practice may be enrolled as a research participant in the 
study).* 
 
1.5 
Study Modification/Discontinuation  
 
The study may be modified or discontinued at any time by the IRB, HRSA, the AIR-P, the 
OHRP, or other government agencies as part of their duties to ensure that research subjects 
are protected. 
 
2. 
BACKGROUND 
2.1 
Rationale 
 
ASD is associated with significant impairments in social, communication and behavioral 
domains and medical and psychiatric comorbidities [1-3]. Unmanaged comorbid conditions 
contribute to increased stress and burden for families [4-5]. Children with ASD have 
increased risk for unmet healthcare needs, which is exacerbated among underserved 
populations [6-8]. Individuals in rural areas face socioeconomic and geographic barriers to 
accessing health care, including high rates of poverty [9]. They are more likely to lack health 
Insurance [10]. Rural areas have significant shortages of specialists, necessitating long 
distance travel for families to access services [10]. Rural children have higher rates of unmet 
medical and dental needs and of emergency department visits than do non-rural populations 
[11].  
 
With increasing prevalence of ASD, diagnostic and treatment demands far exceed the 
capacity of specialty centers [12]. Early identification, referral and effective treatment are 
essential for enhancing outcomes, yet children with ASD face delays in diagnosis [13]. 
Although PCPs provide immediate and community-based care for children, they often feel ill-
equipped to care for children with ASD [14-15]. Children with ASD experience co-occurring 
conditions that PCPs could manage, most notably sleep problems and constipation [3, 16-
17]. However, PCPs report lack of knowledge and confidence in treating children with ASD, 
with resulting unmet healthcare needs [14-15]. PCPs also may lack the knowledge to 
manage ongoing psychotropic medication use. Lack of PCP comfort caring for children with 
ASD likely contributes to higher rates and duration of hospitalizations, greater expenditures, 
and greater use of psychotropic medications in children with ASD [18-21]. Thus, there is a 
critical need to improve early identification of ASD by PCPs and to enhance PCP 
effectiveness in managing sleep, constipation, psychotropic medications and other co-
morbidities for children with ASD. 
 
Extension for Community Healthcare Outcomes (ECHO) was designed to build local 
healthcare capacity and improve access to best practice care for minorities and underserved 
rural populations in New Mexico. ECHO represents an innovative telemedicine-based platform 
connecting local PCPs with specialists at academic medical centers during weekly ECHO 
clinics, and providing education in best-practice treatment protocols, case-based learning, and 
                                            
 
*Providers who are not eligible to participate in the study, but who express interest in 
participating in ECHO Autism will be invited to join the Missouri ECHO Autism Clinic (an open-
enrollment ECHO Clinic that does not currently include a research component). 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 11 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
co-management [22]. The theoretical underpinnings of the model include well-established 
learning theories, all of which emphasize the need for collaborative learning, coaching, and 
mentorship from both experts and peers [23-26]. 
  
By equipping community-based providers to provide best-practice care, the ECHO model 
disseminates academic and specialty knowledge directly into families’ communities. Mainly 
used to date for adult chronic conditions in rural communities, it has proven particularly 
effective with underserved and culturally-diverse populations. ECHO accelerates the 
adoption of effective interventions, guidelines, tools and systems management approaches 
to community practice, helping patients to receive care in their own communities. The ECHO 
model has demonstrated effectiveness in improving both provider self-efficacy and patient 
outcomes for Hepatitis C (HCV), and has now expanded to address other complex medical 
conditions, including rheumatoid arthritis, chronic pain, HIV/AIDS, addiction, and psychiatric 
problems, with rapid expansion into other geographic areas [22, 27-28]. 
 
2.2 
Supporting Data 
 
The results of an initial pilot study of the ECHO Autism intervention support its feasibility and 
effectiveness.  The pilot project included development and implementation of the 6-month 
ECHO Autism curriculum, consisting of 2-hour clinics occurring twice per month, with a 
specific focus on 1) screening and identification of ASD symptoms and 2) management of 
medical and psychiatric comorbidities.  Participants (n = 14 PCPs) completed measures of 
practice behavior and self-efficacy in screening and management of children with ASD at 
baseline (pre-test) and after 6 months of participation in ECHO Autism (post-test).The results 
revealed statistically significant pre- to post-test improvements in self-efficacy in all areas of 
ASD screening and management, in adherence to AAP autism screening guidelines, and in 
use of ASD-specific resources.  Participants also reported high satisfaction with the program. 
 
2.3 
Risks and Benefits  
 
2.3.1 Risks 
 
There are minimal anticipated risks to PCPs for participation in the ECHO Autism 
intervention.  The focus of the intervention is on improving knowledge and practice 
behavior through education and mentorship of PCP participants.  No health-related or 
sensitive information about participants will be collected.  Participant responses and 
data collected during study assessments will be de-identified.  
 
2.3.2 Benefits  
 
Participants will receive a 6-month intervention focused in improving their own 
knowledge, confidence and competence in managing children with ASD in their 
respective practices.  Participants will receive direct benefits in the form of knowledge 
gained and continuing medical education credit.  Participants may also expect to 
benefit from taking part of this research to the extent that they are contributing to the 
development and evaluation of new training methods, and that the information learned 
will benefit other PCPs and the children they serve in the future.  Ultimately, the results 
of the study will benefit children with autism and their families by enhancing access to 
best-practice medical care in local communities. 

Autism Intervention Research  
CONFIDENTIAL 
Page 12 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
3. 
OBJECTIVES 
 
3.1 
Study Objectives: To determine whether participation in a collaborative telehealth 
intervention will result in improved learning, clinical practice behavior and efficacy 
among primary care providers (PCPs).  
 
Hypothesis 1: Following participation in ECHO Autism, PCPs will demonstrate 
significant improvements in ASD knowledge as assessed by pre- to post-
intervention knowledge tests in ASD screening and identification and 
assessment and treatment of medical co-morbidities.  
 
Hypothesis 2: Following participation in ECHO Autism, PCPs will demonstrate 
significant improvements in clinical practice/behavior as assessed by pre- to 
post-intervention chart reviews in ASD screening (co-primary outcome) and 
treatment of medical co-morbidities, in particular, sleep problems and 
constipation (co-primary outcome). 
 
Hypothesis 3: Following participation in ECHO Autism, PCPs will demonstrate 
significant improvements in self-efficacy in ASD screening and identification 
and treatment of medical co-morbidities as assessed by pre- to post- 
intervention tests. 
 
3.2 
Study Outcome Measures  
 
3.2.1 Primary outcome measures 
 
1) ASD Knowledge will be assessed at T1, T2, T3, and T4 using a 33-item 
test developed specifically for the current study. The test is detailed in 
section 8.  
 
2) Clinical Practice/Behavior will be assessed at T1, T2, T3, and T4 by 
review of a subset of charts from each PCP’s practice.  Five subsets of 
charts will be reviewed. The charts that will be reviewed are detailed in 
section 8.  
 
3) Self-Efficacy will be assessed at T1, T2, T3, and T4 using a questionnaire 
developed for a previous ECHO Autism pilot study.  The questionnaire is 
detailed in section 8.  
 
3.2.2 Secondary measures  
 
1) Demographic and Practice Information will be collected at T1 using a 
demographic questionnaire.  Providers will report the following information: 
age, gender, race, ethnicity, zip code of practice, patient population 
(volume, patient characteristics), years of practice, provider type, and 
previous training in ASD.  
 
2) Satisfaction will be assessed at T3 using a 12-item survey developed for a 
previous ECHO Autism pilot study.  The survey includes 10 questions 

Autism Intervention Research  
CONFIDENTIAL 
Page 13 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
assessing overall satisfaction with participation in the ECHO Autism clinic 
(rated on a 5-point Likert-type scale), and two questions asking for overall 
comments and suggestions. 
 
3) Perceived Barriers to caring for children with autism in primary care will be 
assessed at T1, T2, T3, and T4 by participant response to an 11-item 
checklist. 
 
4) Participation in each ECHO session for a PCP will be abstracted from the 
CME sign in sheets. 
 
5) Precise schedule of ECHO topics and dates of PCP chart reviews will be 
collected and considered as potential explanatory values for modeling 
outcomes, especially for the T2 assessment which occurs during the ECHO 
intervention. 
 
4. 
STUDY DESIGN  
4.1 
Overall Study Design and Plan 
 
A cluster-randomized design will be used, with sequential, staggered roll-out of the ECHO 
Autism intervention to 5 clusters of participants over a 1 year period. This design was 
chosen to maximize our ability to determine effectiveness of the intervention while 
minimizing potential contamination across groups and addressing potential ethical concerns. 
First, it would be problematic to randomize at the level of individual participants due to 
potential contamination across groups. Second, the staggered roll-out allows for comparison 
of each cluster to contemporaneous control groups as well as to its own baseline. Lastly, 
because of the anticipated benefit of the intervention for underserved populations, 
randomization of participants into a control group that does not receive the intervention 
would be unethical. The chosen design addresses these concerns by allowing us to 
randomize at the level of cohort (ECHO Autism Hubs), implement the intervention to cohorts 
in a stepwise manner, and test the effectiveness of the intervention using multiple data 
collection points under both baseline and ECHO conditions.  
 
 
 
12/1/2016 
3 Months 
3/1/2017 
3 months 
6/1/2017 
3 months 
9/1/2017 
3 months 
12/1/2017 
3 months 
3/1/2018 
3 months 
6/1/2018 
3 months 
9/1/2018 
Cohort 1 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
 
 
 
 
 
 
 
Cohort 2 
 
 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
 
 
 
 
 
Cohort 3 
 
 
 
 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
 
 
 
Cohort 4 
 
 
 
 
 
 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
 
Cohort 5 
 
 
 
 
 
 
 
 
T1 
ECHO 
T2 
ECHO 
T3 
 
T4 
 
 
 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 14 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
4.2 
Study Centers  
 
The study will be implemented at 10 different ATN Sites (ECHO Autism Hubs):  15 PCP 
participants per ECHO Autism Hub will be recruited from the geographic region in which the 
ATN site is located. 
 
1. Children’s Hospital of Philadelphia 
2. Lurie Center for Autism 
3. University of Pittsburgh Medical Center 
4. University of Rochester 
5. Cincinnati Children’s Hospital Medical Center 
6. Nationwide Children’s Hospital 
7. Arkansas Children's Hospital/UAMS 
8. Vanderbilt University Medical Center 
9. The Center for Autism & Neurodevelopmental Disorders at UC Irvine 
10. Toronto ATN Site (Holland Bloorview Kids Rehab) 
 
4.3 
Study Enrollment Procedures 
 
4.3.1 Recruitment of Participants 
 
Each ECHO Autism Hub will utilize a number of different recruitment strategies to 
recruit 15 PCP participants who meet inclusion criteria (as described previously).  The 
Massachusetts League of Community Health Centers will work closely with each 
ECHO Autism Hub to facilitate recruitment efforts.  Each ECHO Autism Hub team may 
generate a list of potential PCP practices based on publically available searchable 
databases. PCP practices may meet federal designation in any of the following: 
 
Primary Care Health Professional Shortage Areas (PC-HPSAs) 
 
Medically Underserved Areas and Populations (based on the Index of Medical 
Underservice) 
 
Federally Qualified Health Centers (FQHCs) 
 
 PCP practices are not required to meet these federal designations in order to be 
eligible for the study. 
 
ECHO Autism Hub staff may then contact potential physicians via a recruitment letter 
or email that includes a link to ECHO Autism web-based resources. The letter can be 
followed by phone calls and/or face-to-face recruitment strategies as appropriate. 
Once a PCP agrees to participate, study staff will re-evaluate eligibility prior to 
consenting (based on inclusion/exclusion criteria described above).   
 
Screening logs will be maintained to track recruitment efforts and results, including 
number of potentially eligible participants contacted, number of interested 
participants, and results of initial screening (reasons for ineligibility, reasons for 
nonparticipation of eligible participants). 
 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 15 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
Additional recruitment strategies may include: 
 
Attendance and/or presentations at local meetings such as the state 
chapters of the American Academy of Pediatrics (AAP), American Academy 
of Family Physicians, and State Primary Care Associations (FQHC). 
 
Social and traditional media posts 
 
Email recruitment through state-wide or regional primary care association 
listservs  
 
4.3.2 Consent and Assent  
 
A waiver of consent is being requested for the purpose of this study as this 
research involves no more than minimal risk. The waiver will not adversely affect the 
rights and welfare of the participants. Participants will be given a recruitment letter that 
outlines the research study for their review. A copy of this document is included as 
Appendix B. Participants will provide consent by their participation in the 
baseline/preintervention battery of assessments.  
 
4.4 
Study Duration 
 
As noted above, a cluster-randomized design will be used, with sequential, staggered roll-
out of the ECHO Autism intervention to 5 clusters over a 1 year period.  The intervention will 
be delivered at 10 ECHO Autism Hubs, two being randomly assigned to each of the 5 
clusters.  Cluster assignment will determine the timing of the intervention start-date for each 
ECHO Autism Hub.  Each ECHO Autism Hub will deliver the intervention to 15 PCP 
participants (for a total of 150 total participants).  Participants will be assessed at four time 
points (as described below). 
 
Duration of Assessment and Follow-up 
Each PCP participant will complete a battery of assessments at four time points:  
Baseline/Pre-Intervention (T1), Mid-Intervention (T2), Post-Intervention (T3), and Follow-up 
(T4).  The duration of the ECHO intervention will be 6 months, and the interval between each 
assessment point will be 3 months.   
 
4.5 
Participant Remuneration  
 
 Participants will receive $100 after completion of each time point.  
 
4.6 
Protocol Adherence  
 
All ECHO Autism Hub teams will adhere to the procedures outlined in the Protocol, and will 
also adhere to the intervention implementation procedures described in the ECHO Autism 
Manual of Procedures throughout implementation of the 6-month ECHO Autism intervention. 
 
5. 
STUDY ENROLLMENT AND WITHDRAWAL 
5.1 
Number of Study Subjects  
 
Participants will include 150 primary care providers.  Inclusion/exclusion, recruitment, 
consent, and enrollment are described in the sections below. 

Autism Intervention Research  
CONFIDENTIAL 
Page 16 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
 
5.2 
Inclusion and Exclusion Criteria  
 
5.2.1 Subject Inclusion Criteria  
 
 
Current practice as a primary care provider (PCP).  
 
Currently providing care for children.   
 
Professional training in:  general pediatrics, family medicine, advance practice 
nursing (i.e. nurse practitioner or physician assistant).   
 
Active medical license in the state of practice. 
 
Patient population is at least 50% underserved. 
 
5.2.2 Subject exclusion criteria  
 
Trainee status (e.g., medical student, intern, resident, or other pre-professional 
trainee). 
 
Subspecialist (e.g., psychiatrists, neurologists, developmental and behavioral 
pediatricians). 
 
Practicing within the same practice as another PCP Participant (i.e., only one PCP 
participant from any given practice may be enrolled as a research participant in the 
study). 
 
5.3 
Treatment Assignment Procedures  
 
5.3.1 Randomization procedures  
 
The intervention will be delivered at 10 ECHO Autism Hubs, two being randomly 
assigned to each of the 5 clusters.  Cluster assignment will determine the timing of the 
intervention start-date for each ECHO Autism Hub.  Each ECHO Autism Hub will 
deliver the intervention to 15 PCP participants (for a total of 150 participants).  
Randomization of cluster assignment was completed by the DCC on 1/12/2016. 
 
5.3.2 Reasons for withdrawal 
 
Participation will be voluntary.  Participants will not be withdrawn from the study unless 
they request to discontinue participation. 
 
5.3.3 Handling of withdrawals  
 
Data from withdrawn participants will be stored with data from participants who complete 
the study.  No further data will be collected from participants who have withdrawn, and 
participant decisions to withdraw will be noted in the data collection system. 
 
6. 
STUDY INTERVENTIONS 
6.1 
Interventions, Administration and Duration 
 
During the intervention phase, each ECHO Autism Hub team will provide twice-monthly 2-
hour ECHO Autism Clinics for 15 PCP participants during a 6-month period. Each ECHO 
Autism Hub will be comprised of a team of up to 5 autism specialists (Physician/Autism 
Specialist, Psychologist, Family Navigator, Dietician, and Parent Expert).  

Autism Intervention Research  
CONFIDENTIAL 
Page 17 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
 
Each Clinic will utilize high quality, secure video conferencing technology to allow PCPs to 
interface with the ECHO Autism Hub team and all other participants, view documents, and 
view videos on screen (with minimal technological requirements for participants). The 
intervention will follow the protocol previously developed and tested by the ECHO Autism 
Leadership team. Based on this protocol, each ECHO Autism Clinic will include a didactic 
presentation, 2 to 3 PCP-generated case presentations, expert feedback, and group 
discussion. Although the ECHO Clinic will include discussion of specific cases, no 
identifiable personal health information will be shared, individual patients will not be 
identified, and no direct patient care will be provided (PCP participants will maintain 
responsibility for care of their patients, but will develop new clinical skills through guided 
practice and collaborative learning). ECHO Autism didactic presentations will include use of 
AIR-P toolkits, guidelines, and algorithms to enhance medical care of children with ASD, with 
particular emphasis on identification/screening and management of co-morbid conditions. 
This combination of case-based learning, co-management with autism specialists, and 
didactic presentations provides multiple learning modalities for enhancing PCP knowledge 
and expertise. 
 
Each team will adhere to the intervention procedures described in the Manual of Procedures. 
The ECHO Autism Leadership team at the University of Missouri (MU) and the Replication 
Support Team at the University of New Mexico will train and mentor each ECHO Autism Hub 
in delivery and implementation of the intervention. 
 
6.2 
Adherence Assessment 
 
Adherence/fidelity to the ECHO model will be assessed using a 25-item observer-rated form 
assessing fidelity of implementation including: training flow, facilitator engagement of 
participants, and other indicators of adherence. The measure was developed by the UNM 
ECHO Team to ensure that facilitators adhere to the model. Fidelity will be assessed by 
project leadership observing 3 randomly selected Clinics for each ECHO Autism Hub. 
 
7. 
STUDY SCHEDULE 
7.1 
Screening  
 
All providers who express interest in participation in ECHO Autism following initial 
recruitment efforts will be screened to ensure eligibility for the study. All providers who are 
screened will be tracked in a screening log maintained by each HUB. Screening will be 
conducted by phone, email, and a web-search of the state’s medical accreditation agency to 
ensure that the provider meets the following criteria: 
 
 
Current practice as a primary care provider (PCP). 
 
Currently providing care for children.   
 
Professional training in:  general pediatrics, family medicine, advanced practice nursing 
(i.e. nurse practitioner or physician assistant).  
 
Active medical license in state of practice.   
 
Patient population is at least 50% underserved. 
 
7.1.1 Screen Failures  
Providers who are not eligible to participate in the study, but who express interest in 
participating in ECHO Autism will be invited to join the Missouri ECHO Autism Clinic 

Autism Intervention Research  
CONFIDENTIAL 
Page 18 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
(an open-enrollment ECHO Clinic that does not currently include a research 
component). 
 
7.2 
Assessments 
 
Each PCP participant will complete the battery of provider-completed measures (Section 
7.2.1) at four time points:  Baseline/Pre-Intervention (T1), Mid-Intervention (T2), Post-
Intervention (T3), and Follow-up (T4).  The duration of the ECHO intervention will be 6 
months. The target time point for the T2 assessment is between the 6th and 7th ECHO 
sessions. The T3 assessment will occur within 4 weeks of completion of the final ECHO 
session. A final assessment will be conducted between 9 and 10 months after the start of the 
ECHO program.  
 
In addition, chart reviews (Section 7.2.2) will be done in the same time frame for T1, T3, and 
T4.   Because it would not be feasible to do the chart review in the two weeks for 15 
participants, the T2 review will include charts from the 30 or 60 days before the 7th ECHO 
session for all participants. 
 
Measures are listed below and described in section 8.  
 
7.2.1 Provider-Completed Measures 
 
1) ASD Knowledge assessed by a 33-item test of knowledge of ASD 
screening/identification, psychiatric co-morbidities, medical co-morbidities, 
and management of additional ASD-specific needs.   
 
2) Self-Efficacy assessed by a 57-item questionnaire of self-efficacy in ASD 
screening, identification, and management of medical and psychiatric 
comorbidities.  
 
3) Demographic and Practice Information assessed by self-report 
questionnaire at T1 
 
4) Satisfaction will be assessed using a 12-item self-report survey at T3. 
 
5) Perceived Barriers to caring for children with autism in primary care will be 
assessed at T1, T2, T3, and T4 by participant response to an 11-item 
checklist. 
 
 
7.2.2 Chart Review Measures 
 
1) Clinical Practice/Behavior will be assessed by review of a subset of charts 
from each PCP’s practice.  Study staff will complete chart reviews either in-
person at each practice location or by remote review using an electronic 
medical record. Five subsets of charts will be reviewed with no more than 
25 charts from each subset being reviewed. The charts that will be reviewed 
are detailed in section 8.  
 
 
 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 19 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
7.2.3 Process Measures 
 
1) Participation in each ECHO session for a PCP will be abstracted from the 
CME sign in sheets. 
 
2) Precise schedule of ECHO topics and dates of PCP chart reviews will be 
collected and considered as potential explanatory values for modeling 
outcomes, especially for the T2 assessment and chart review which occurs 
during the ECHO intervention. 
 
7.3 
Protocol Deviations 
 
Any deviations from the protocol must be recorded in the research record and reported to 
the ECHO Autism Leadership team, the DCC, and the appropriate IRB. 
 
8. 
CLINICAL ASSESSMENTS AND OUTCOME MEASURES  
 
8.1 
Primary Outcome Measures 
 
1) ASD Knowledge will be assessed at T1, T2, T3, and T4 using a 33-item test developed 
specifically for the current study.  The original test was developed and piloted with a group of 
14 PCP participants, questions with very low difficulty were removed and/or reworded (e.g., 
if ≥ 90% of participants answered correctly at pre-test), additional questions were included to 
ensure that all content was adequately covered.  The revised version was then piloted in a 
second sample of nine PCPs.  The test assesses knowledge in the areas of ASD 
screening/identification, psychiatric co-morbidities, medical co-morbidities, and management 
of additional ASD-specific needs.   
 
2) Clinical Practice/Behavior will be assessed at T1, T2, T3, and T4 by review of a subset of 
charts from each PCP’s practice.  Five subsets of charts will be reviewed, with a limit of 25 
charts in any group.  If more than 25 well-child visits at a specific age are available for chart 
review, the most recent 25 well-child visits at a specific age will be reviewed.  The groups 
are:   
 
 
1.Charts for all children seen for 9-month well-child visits in the 30 days prior to the 
date of chart review (for assessment of   ASD screening and referral practices). 
 
 
2. Charts for all children seen for 18-month well-child visits in the 30 days prior to the 
date of chart review (for assessment of ASD-screening and referral practices). 
 
 
3. Charts for all children seen for 24-month well-child visits in the 30 days prior to the 
date of chart review (for assessment of ASD-screening and referral practices). 
 
 
4. Charts for all children seen for 30-month well-child visits in the 30 days prior to the 
date of chart review (for assessment of ASD-screening and referral practices).  
 
 
5. Charts for all children with ASD in the 60 days prior to the date of chart review (for 
assessment comorbidity management). 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 20 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
3) Self-Efficacy will be assessed at T1, T2, T3, and T4 using a questionnaire developed for a 
previous ECHO Autism pilot study.  The questionnaire is comprised of 57 items across five 
domains:  1) ASD screening and identification (7 items), 2) ASD referral and resources (9 
items), 3) assessment and treatment of medical comorbidities (19 items), 4) assessment and 
treatment of psychiatric comorbidities (13 items), and 5) Additional (9 items). Participants 
report the degree to which they are confident in their ability to provide effective care in each 
domain.  Items are rated on a 6-point Likert-type scale (ranging from 1= “no confidence” to 6 
= “highly confident/expert”). 
 
8.2 
Secondary Measures  
 
1) Demographic and Practice Information will be collected at T1 using a demographic 
questionnaire.  Providers will report the following information: age, gender, race, ethnicity, 
zip code of practice, patient population (volume, patient characteristics), years of practice, 
provider type, and previous training in ASD.  
 
2) Satisfaction will be assessed at T3 using a 12-item survey developed for a previous ECHO 
Autism pilot study.  The survey includes 10 questions assessing overall satisfaction with 
participation in the ECHO Autism clinic (rated on a 5-point Likert-type scale), and two 
questions asking for overall comments and suggestions. 
 
3) Perceived Barriers to caring for children with autism in primary care will be assessed at T1, 
T2, T3, and T4 by participant response to an 11-item checklist 
 
4) Participation in each ECHO session for a PCP will be abstracted from the CME sign in 
sheets. 
 
5) Precise schedule of ECHO topics and dates of PCP chart reviews will be collected and 
considered as potential explanatory values for modeling outcomes, especially for the T2 
assessment which occurs during the ECHO intervention. 
 
8.3 
Intervention Fidelity Evaluations  
 
1) ECHO Fidelity will be assessed using a 25-item observer-rated form assessing fidelity of 
implementation including: training flow, facilitator engagement of participants, and other 
indicators of adherence. The measure was developed by the UNM ECHO Team to ensure 
that facilitators adhere to the model. Fidelity will be assessed at 3 randomly selected Clinics 
for each ECHO Autism Hub. 
 
9. 
STATISTICAL ANALYSIS PLAN  
 
9.1 
Statistical Considerations  
 
9.1.1 Data Analysis  
 
There are two co-primary endpoints (the percent of children being screened for autism, 
with the chart reviews for children of all ages combined; and the percent of co-
morbidity management of autistic children in the practice).  To preserve the overall 
study Type I error at 0.05, we will use an alpha-level of 0.025 for each of the two 
endpoints separately.    

Autism Intervention Research  
CONFIDENTIAL 
Page 21 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
 
Standard summary statistics (e.g. median/IQR) will be calculated separately for each 
center (ECHO HUB) and PCP within center separately for each time point.   The 
results will be presented separately for each center over time using a spaghetti plot.  
Graphs will also be prepared to present the results of each PCP within a center.   
 
The primary outcome analysis will use a generalized mixed model analysis, using a 
binary distribution and logit link for each outcome (patient screened / not screened or 
patient received / did not receive appropriate co-morbidity management).  Treatment 
effect and trend over calendar time will be fixed effects in the model.  The model will 
include center, PCP within center, and nominal study period of the observation as 
random effects.  The primary analysis will use data from T1 (baseline) and T3 (post-
intervention).  The determination of the utility of the ECHO intervention will be based on 
these results. 
 
An additional analysis of both primary endpoints will use data from T1, T2, and T3.  
The model described above will be expanded to incorporate the precise timing of the 
T2 assessment for the PCP and allow for the estimated impact of the ECHO training 
through that time point on each of the measures separately.   For example, the change 
in co-morbidity management at the T2 assessment would likely depend on the number 
of sessions devoted to the topic prior to the T2 assessment for the specific PCP.  
 
For the screening endpoint, a secondary analysis will include a factor for age group 
and age group x treatment interaction to determine whether screening changes were 
related to the age group.  
 
 
 
 
 
 
 
 
 
One complication with our approach is that if a PCP practice has no autism patients at 
the start of the study, then the change in practice for autism patients over time will be 
uninformative.   We anticipate that this will occur in very few practices, but we will 
perform an additional analysis of the number of autism patients in each practice using 
a similar approach but using a Poisson distribution and log link in the analysis. 
 
A similar analysis approach to the primary analysis will be used for the other endpoints 
collected over time (e.g. ASD knowledge and self-efficacy measures), with appropriate 
adjustment of the model for the distribution of the outcome variable.  The primary 
analysis for these endpoints will use results from T1, T2, and T3 only, and use the time 
on intervention as the estimate of the predicted treatment effect for period T2. 
 
An additional analysis of the primary endpoints, ASD knowledge, and self-efficacy will 
use data from T3 and T4 only.  The purpose of this analysis is to determine if there is a 
practice / skills / self-efficacy decline after the ECHO program ends.   
   
For data available at only a single time-point (e.g. satisfaction), results will be 
summarized separately for each center and compared across centers using a Kruskal-
Wallis test.    
 
Exploratory analyses will consider the effect of PCP demographic and practice 
information on the treatment effect on the primary endpoints. 
 
 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 22 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
 
9.1.2 Power Considerations  
 
Given the complexity of the proposed analysis, power calculations were based on 
simulations.  The data generation process allowed for random effects for center, PCP 
within center, and nominal period.  There was no time trend in the data, although a 
potential time trend as a fixed effect was included in the model.   Simulations were 
done for 10 randomly selected seeds (from several different websites and different 
random number tables), 1000 simulations per seed.  The data generating process 
allowed for approximately a 50% intra-class correlation for the PCP within group effect, 
reflecting the possibility that the impact of ECHO would be correlated within each 
center, even with good fidelity to the intervention program.  Simulations allowed for 
varying numbers of patients per PCP practice.   
 
If there are on average 5 patients per PCP (e.g. 5 autistic patients seen in the last 60 
days), we would have over 90% power to detect an increase of 15% in appropriate co-
morbidity management (alpha=0.025, two-sided).  If there are 15 patients per PCP on 
average (e.g. 15 patients with well child visits in the past month), we would have over 
90% power to detect an increase of 10% in autism screening (alpha=0.025, two-sided).  
If the number of patients per PCP was higher, then we would have over 90% power for 
even smaller differences.  Results were consistent for the different seeds.   
 
 
10. 
DATA COLLECTION, MANAGEMENT, AND MONITORING 
 
10.1 
Role of Data Management 
 
10.1.1 Web-Based Data Collection and Management System 
 
Data collection will occur via a web-based data entry system to allow easy access to 
enrollment 24 hours a day, seven days a week. Participants will complete assessments 
using an online portal.  
 
10.1.2 Certification in the Use of Web-Based Data Entry System  
 
The DCC will provide training and certification of study staff in the use of the data entry 
system.  Once certified, users are permitted to enter data into the production system.  
Access is password controlled. Certification for use of the web-based data entry 
system will be completed via individual practicum assessment. 
 
Participants will be trained by study coordinators in the use of the EDC portal.  
 
10.1.3 Data Entry and Checks 
 
Data for individual participants will be recorded on electronic case report forms (eCRF) 
in an electronic data capture system.  All participants screened for the study, including 
the screen failures, must be entered into the system. The EDC will reflect participant 
status (screen failure, enrolled, early termination, completed) at each phase 
during the course of the study.  Participants will not be identified on the eCRFs by 
name or initials. Each participant will be assigned a study identification number.   
 

Autism Intervention Research  
CONFIDENTIAL 
Page 23 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
Clinical data processing and management will be employed based on the procedures 
developed in conjunction with the AIR-P DCC. All of the data entered into the electronic 
data capture system will be checked for valid values and ranges, between-item logical 
consistency, and within-subject variation. 
 
10.1.4 Quality Assurance  
 
Prior to the initiation of the study, an investigator’s meeting will be held with the AIR-P 
CCC, AIR-P DCC, the investigators and their study coordinators.  This meeting will 
include a detailed discussion of the protocol, performance of study procedures, safety 
reporting requirements, electronic data capture system training and eCRF completion 
and simulation of study procedures, as applicable.  Study staff that is responsible for the 
collection and submission of the study data will be required to pass eCRF training for 
certification prior to use of the production system for submission of the data.  The study 
Manual of Procedures will be reviewed during training for site personnel and should be 
utilized to reference key details regarding study processes. 
 
After completion of the entry process, computer logic checks or Integrity reports will be 
executed to assess data inconsistencies (e.g., inconsistent study dates).  A response to 
these reports is required from site personnel by the defined report date. In addition, data 
modifications to the data field(s) must be made in the electronic data capture system 
which tracks the audit history of all data entered and modified.  
 
10.2 
Data Handling and Record Keeping 
 
10.2.1 Confidentiality 
 
Raw data will be stored in locked cabinets in a locked office at each site. All evaluation 
forms, reports and other records that leave a site will be identified only by the Study 
Identification Number (SID) to maintain participant confidentiality. De-identified data will 
be submitted to a central, password-protected database provided by the DCC. The key 
connecting participants to their SID will be secured in a locked cabinet at each site. All 
computer entry and networking programs will be done using SIDs only.  Data forms will 
only be identified by SID. A limited personal identifier (email address) will be collected 
in the EDC in order for the EDC to send out automated reminders for participants to 
complete the online portal assessments. For participants not wishing to provide this, 
the site will be responsible for contacting participants about completing the online 
portal assessments. The database will not contain any other personal identifiers other 
than study identification number. 
 
ECHO clinics utilize secure video conferencing technology that meets VTC industry 
standards H.264, H.265, H.239, H.323 and SIP. Although the ECHO Clinic will include 
discussion of specific cases, no identifiable personal health information will be shared. 
Chart reviews will be conducted on-site by study staff. For research purposes, the 
results from the record reviews (for chart reviews groups 1 to 4: the number of 
individuals screened / well-child visits by age group; for chart reviews group 5: the 
number of individuals receiving co-morbidity management / number of ASD visits in past 
60 days will be summarized from these source data. PHI will be removed and records 
will be identified with a unique identifier generated by Project ECHO. Study staff will be 
trained on the importance of confidentiality and HIPAA requirements. 

Autism Intervention Research  
CONFIDENTIAL 
Page 24 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
 
10.2.2 Retention of records  
 
Sites will comply with their individual IRB’s policies for retention of records.  
 
 
10.2.3 Publications  
 
Publication of the results of this trial will be governed by the policies and procedures 
developed by the Executive Committee. Any presentation, abstract, or manuscript will be 
made available for review by AIR-P and HRSA prior to submission.
 
 
10.2.4 Data Sharing 
 
Limited de-identified data will be shared with the ECHO Institute. The ECHO Institute at 
the University of New Mexico developed the ECHO model that is used in the ECHO 
Autism study. The shared data will include participating provider’s practice zip codes 
which will not be linked to the participant’s study ID or any other study data.  

Autism Intervention Research  
CONFIDENTIAL 
Page 25 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
11. 
REFERENCES 
 
1. 
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 
Fifth Edition (DSM-5). Washington, DC: American Psychiatric Association; 2013. 
2. 
Myers SM, Johnson CP. Management of children with autism spectrum disorders. 
Pediatrics. 2007;120(5):1162-1182. 
3. 
Simonoff EM, Pickles AP, Charman TP, Chandler SP, Loucas TOMP, Baird GF. 
Psychiatric disorders in children with autism spectrum disorders: Prevalence, 
comorbidity, and associated factors in a population-derived sample. Journal of the 
American Academy of Child & Adolescent Psychiatry. 2008;47(8):921-929. 
4. 
Estes A, Munson J, Dawson G, Koehler E, Zhou XH, Abbott R. Parenting stress and 
psychological functioning among mothers of preschool children with autism and 
developmental delay. Autism. 2009;13(4):375-387. 
5. 
Zablotsky B, Kalb LG, Freedman B, Vasa RA, Stuart EA. Health care experiences and 
perceived financial impact among families of children with an autism spectrum disorder. 
Psychiatric Services. 2014;65(3):395-398. 
6. 
Kogan MD, Strickland BB, Blumberg SJ, Singh GK, Perrin JM, van Dyck PC. A national 
profile of the health care experiences and family impact of autism spectrum disorder 
among children in the United States, 2005-2006. Pediatrics. 2008;122(6):e1149-1158. 
7. 
Liptak GS, Stuart T, Auinger P. Health care utilization and expenditures for children with 
autism: Data from U.S. national samples. J Autism Dev Disord. 2006;36(7):871-879. 
8. 
Gurney JG, McPheeters ML, Davis MM. Parental report of health conditions and health 
care use among children with and without autism: National Survey of Children's Health. 
Archives of Pediatric Adolescent Medicine. 2006;160(8):825-830. 
9. 
US Department of Agriculture Economic Research Service. Rural america at a glance. 
http://www.ers.usda.gov/Publications/EIB85/EIB85.pdf. Updated 2011. 
10. 
National Center for Health Statistics. Health, united states, 2010: With special feature on 
death and dying. Hyattsville, MD: National Center for Health Statistics; 2011.  
11. 
DeVoe JE, Krois L, Stenger R. Do children in rural areas still have different access to 
health care? results from a statewide survey of oregon's food stamp population. J Rural 
Health. 2009;25(1):1-7.  
12. 
Centers for Disease Control and Prevention. Prevalence of autism spectrum disorder 
among children aged 8 years — autism and developmental disabilities monitoring 
network, 11 sites, united states, 2010. 
http://www.cdc.gov/mmwr/preview/mmwrhtml/ss6302a1.htm?s_cid=ss6302a1_w. 
Accessed 4/14/2015, 2015 
13. 
Mazurek MO, Handen BL, Wodka EL, Nowinski L, Butter E, Engelhardt CR. Age at first 
autism spectrum disorder diagnosis: The role of birth cohort, demographic factors, and 
clinical features. J Dev Behav Pediatr. 2014;35(9):561-569. 
14. 
Carbone PS. Moving from research to practice in the primary care of children with 
autism spectrum disorders. Acad Pediatr. 2013;13(5):390-399.  
15. 
Golnik A, Maccabee-Ryaboy N, Scal P, Wey A, Gaillard P. Shared decision making: 
Improving care for children with autism. Intellectual & Developmental Disabilities. 
2012;50(4):322-331. 
16. 
Wang LW, Tancredi DJ, Thomas DW. The prevalence of gastrointestinal problems in 
children across the united states with autism spectrum disorders from families with 
multiple affected members. J Dev Behav Pediatr. 2011;32(5):351-360.  
17. 
Richdale AL. Sleep problems in autism: Prevalence, cause, and intervention. Dev Med 
Child Neurol. 1999;41(1):60-66.  

Autism Intervention Research  
CONFIDENTIAL 
Page 26 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
18. 
Liptak GS, Stuart T, Auinger P. Health care utilization and expenditures for children with 
autism: Data from U.S. national samples. J Autism Dev Disord. 2006;36(7):871-879.  
19. 
Mandell DS, Cao J, Ittenbach R, Pinto-Martin J. Medicaid expenditures for children with 
autistic spectrum disorders: 1994 to 1999. J Autism Dev Disord. 2006;36(4):475-485.  
20. 
Lokhandwala T, Khanna R, West-Strum D. Hospitalization burden among individuals 
with autism. J Autism Dev Disord. 2012;42(1):95-104.  
21. 
Gurney JG, McPheeters ML, Davis MM. Parental report of health conditions and health 
care use among children with and without autism: National survey of children's health. 
Arch Pediatr Adolesc Med. 2006;160(8):825-830.  
22. 
Arora S, Kalishman S, Thornton K, et al. Expanding access to hepatitis C virus 
treatment--extension for community healthcare outcomes (ECHO) project: Disruptive 
innovation in specialty care. Hepatology. 2010;52(3):1124-1133.  
23. 
Bandura A. Social foundations of thought and action. Englewood Cliffs, NJ: ; 1986.  
24. 
Bandura A. Self-efficacy: The exercise of control. New York, NY: Freeman; 1997.  
25. 
Vygotsky L. Mind and society: The development of higher mental processes. Cambridge, 
MA: Harvard University Press; 1978.  
26. 
Lave J, Wenger E. Situated learning: Legitimate peripheral participation. Cambridge 
University Press; 1991.  
27. 
Arora S, Thornton K, Murata G, et al. Outcomes of treatment for hepatitis C virus 
infection by primary care providers. N Engl J Med. 2011;364(23):2199-2207. 
28. 
Scott JD, Unruh KT, Catlin MC, et al. Project ECHO: A model for complex, chronic care 
in the pacific northwest region of the united states. J Telemed Telecare. 2012;18(8):481-
484. 
 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 27 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
APPENDIX A 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation 
Screening 
 
Baseline/Pre-
Intervention 
3 mo  
Mid-
Intervention 
6 mo  
Post-
Intervention 
9-mo 
Continuing 
follow-up 
Informed Consent 
X 
 
 
 
 
Demographics 
Questionnaire 
 
X 
 
 
 
Self-Efficacy 
Questionnaire 
 
X 
X 
X 
X 
ASD Knowledge Test 
 
X 
X 
X 
X 
Chart Review 
 
X 
X 
X 
X 
Perceived Barriers 
Questionnaire 
 
X 
X 
X 
X 
Satisfaction 
Questionnaire 
 
 
 
X 
 
ECHO Fidelity 
 
 
X 
X 
 
Process Measures 
 
 
X 
X 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 28 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
APPENDIX B 
 
 
 
 
Dear Participant, 
 
Please accept this letter as an invitation to participate in a research study being conducted by XX and 
the Autism Intervention Research Network on Physical Health (AIRP), a network funded by the Health 
Resources and Services Administration (HRSA). The purpose of this research is to determine and 
evaluate the effectiveness of the Extension for Community Healthcare Outcomes (ECHO) model in 
providing primary care providers with education specific to Autism Spectrum Disorder.  
 
Study Procedures 
 
You will be asked to participate in twice-monthly ECHO Autism clinics over a period of 6 months. Each 
Clinic will utilize high quality, secure video conferencing technology to allow you to interface with the 
ECHO Autism Hub team and all other participants, view documents, and view videos on screen. There 
will be minimal technological requirements. Each ECHO Autism Clinic will include a didactic 
presentation, 2 to 3 PCP-generated case presentations, expert feedback, and group discussion. 
Although the ECHO Clinic will include discussion of specific cases, no identifiable personal health 
information will be shared, individual patients will not be identified, and no direct patient care will be 
provided. You as the provider will maintain responsibility for care of your patients, but will develop new 
clinical skills through guided practice and collaborative learning.  
 
As part of your participation in the study, you will be asked to complete questionnaires at four time 
points:  before you begin participation in ECHO, after 3 months of participation in ECHO, after 6 months 
of participation in ECHO, and 3 months after the end of ECHO.  All questionnaires will be completed 
online through a secure portal. You will receive $100 after completion of each time point.   
 
Participation in the study also involves on-site or remote chart review by study coordinators at similar 
time points. Remote chart review will be conducted using an electronic medical record. Charts that 
meet the following criteria will be reviewed:  
 
1. Charts for all children seen for 9-month, 18-month, 24-month, and 30-month well-child visits in the 
30 days prior to the date of chart review, up to a maximum of 25 charts in a specific age-group; if more 
than 25 well-child visits occurred in the 30 day interval, the most recent 25 charts will be reviewed; and  
2. Charts for all children with ASD in the 60 days prior to the date of chart review.  
 
All PHI will be removed prior to extraction of chart review data. Study coordinators are trained in HIPAA 
and maintaining patient confidentiality.  
 
Risks and Benefits 
There are minimal anticipated risks for participation in the ECHO Autism intervention.  The focus of the 
intervention is on improving knowledge and practice behavior through education and mentorship of 
primary care providers.  No health-related or sensitive information will be collected.  Your responses 
and data collected during study assessments will be de-identified. 
 
You will benefit from participating in a 6-month program focused in improving your knowledge, 
confidence and competence in managing children with ASD in your practices.  You will receive direct 
benefits in the form of knowledge gained and continuing medical education credit.  You may also 
expect to benefit from taking part of this research to the extent that you are contributing to the 
development and evaluation of new training methods, and that the information learned will benefit 
other PCPs and the children they serve in the future.  Ultimately, the results of the study will benefit 
children with autism and their families by enhancing access to best-practice medical care in local 
communities. 

Autism Intervention Research  
CONFIDENTIAL 
Page 29 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]  
 
Voluntary Participation 
 
Your participation in this research is voluntary. You may decline to participate, or withdraw from the 
study at any time. If you decide to withdraw, no further information will be collected. Data collected 
before withdrawal will be stored with data from participants who complete the study. 
 
Confidentiality 
 
All of your assessment responses will be kept confidential and will be de-identified. Assessments will be 
entered in a secure, password-protected electronic data capture system. Data entered into this system 
will only be accessible by study staff. All ECHO Autism clinics will be conducted via secure video 
conferencing. Information discussed during the clinics will not be shared.  
 
Contact Information 
 
If you have any questions regarding this study, please contact the researchers below: 
 
XX 
 
If you have questions about your rights as a research participant, or wish to obtain information, ask 
questions or discuss any concerns about this study with someone other than the researcher(s), please 
contact the: 
 
IRB 
 
 
Thank you, 
 
XX 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Autism Intervention Research  
CONFIDENTIAL 
Page 30 of 30 
ECHO Autism 
Version [4] 
 
[May 9, 2017]