Evaluating an Adverse Childhood Experience-Targeting Advocate Model of a Substance Use Prevention Program: A Hybrid Type 1 Effectiveness-Implementation Trial
Phase: NA
Conditions: Substance-Related Disorders, Adverse Childhood Experiences
Sponsor: RTI International

 
 
Evaluating an ACE-Targeting Advocate Model of a 
Substance Use Prevention Program: A Hybrid Type 1 
Effectiveness-Implementation Trial 
Protocol Number*: 0218779 (Pro00071669) 
Co-Principal Investigators*: Phillip W. Graham, PhD, Dallas J. Elgin PhD 
Sponsor: RTI International 
Grant Title: Evaluating an ACE-Targeting Advocate Model of a Substance Use 
Prevention Program 
Grant Number*: 1 U01CE003482-01-00 
Funded by: National Center for Injury Prevention and Control, Centers for 
Disease Control and Prevention, U.S. Department of Health and Human Services 
Version Number:  v.1.0 
August 31, 2023 
 
 

 
CONFIDENTIALITY STATEMENT 
This document is confidential communication. Acceptance of this document constitutes agreement by the 
recipient that no unpublished information contained herein will be published or disclosed without prior 
approval of the Principal Investigators or other participating study leadership and as consistent with the 
U.S. Centers for Disease Control and Prevention's (CDC) terms of award.  
 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
 
August 31, 2023 
 
iii 
Table of Contents 
STATEMENT OF COMPLIANCE ...................................................................................................................... 1 
1 
PROTOCOL SUMMARY ....................................................................................................................... 1 
1.1 
Synopsis ................................................................................................................................ 1 
1.2 
Schema ................................................................................................................................. 2 
1.3 
Schedule of Activities ........................................................................................................... 6 
2 
INTRODUCTION ................................................................................................................................. 7 
2.1 
Study Rationale ..................................................................................................................... 7 
2.2 
Background ......................................................................................................................... 11 
2.3 
Risk/Benefit Assessment .................................................................................................... 12 
2.3.1 
Known Potential Benefits .................................................................................... 13 
2.3.2 
Assessment of Potential Risks and Benefits ........................................................ 14 
3 
OBJECTIVES AND ENDPOINTS .......................................................................................................... 15 
4 
STUDY DESIGN ................................................................................................................................. 16 
4.1 
Overall Design ..................................................................................................................... 16 
4.2 
Scientific Rationale for Study Design .................................................................................. 18 
4.3 
Justification for Intervention .............................................................................................. 19 
4.4 
End-of-Study Definition ...................................................................................................... 20 
5 
STUDY POPULATION ........................................................................................................................ 20 
5.1 
Inclusion Criteria ................................................................................................................. 20 
5.2 
Exclusion Criteria ................................................................................................................ 20 
5.3 
Lifestyle Considerations ..................................................................................................... 20 
5.4 
Screen Failures ................................................................................................................... 21 
5.5 
Strategies for Recruitment and Retention ......................................................................... 21 
6 
STUDY INTERVENTION(S) OR EXPERIMENTAL MANIPULATION(S) .................................................. 21 
6.1 
Study Intervention(s) or Experimental Manipulation(s) Administration ........................... 21 
6.1.1 
Study Intervention or Experimental Manipulation Description .......................... 21 
6.1.2 
Administration and/or Dosing ............................................................................. 22 
6.2 
Fidelity ................................................................................................................................ 22 
6.2.1 
Interventionist Training and Tracking .................................................................. 22 
6.3 
Measures to Minimize Bias: Randomization and Blinding ................................................. 22 
6.4 
Study Intervention/Experimental Manipulation Adherence .............................................. 22 
6.5 
Concomitant Therapy ......................................................................................................... 23 
6.5.1 
Rescue Therapy .................................................................................................... 23 
7 
STUDY INTERVENTION/EXPERIMENTAL MANIPULATION DISCONTINUATION AND PARTICIPANT 
DISCONTINUATION/WITHDRAWAL ................................................................................................. 23 
7.1 
Discontinuation of Study Intervention/Experimental Manipulation ................................. 23 
7.2 
Participant Discontinuation/Withdrawal from the Study .................................................. 23 
7.3 
Lost to Follow-Up ............................................................................................................... 23 
8 
STATISTICAL CONSIDERATIONS ....................................................................................................... 24 
8.1 
Statistical Hypotheses ........................................................................................................ 24 
8.2 
Sample Size Determination ................................................................................................ 24 
8.3 
Populations for Analyses .................................................................................................... 26 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
 
August 31, 2023 
 
iv 
8.4 
Statistical Analyses ............................................................................................................. 26 
8.4.1 
General Approach ................................................................................................ 26 
8.4.2 
Analysis of the Primary Endpoint(s) ..................................................................... 31 
8.4.3 
Analysis of the Secondary Endpoint(s) ................................................................ 31 
8.4.4 
Safety Analyses .................................................................................................... 31 
8.4.5 
Baseline Descriptive Statistics ............................................................................. 32 
8.4.6 
Planned Interim Analyses .................................................................................... 32 
8.4.7 
Sub-Group Analyses ............................................................................................. 32 
8.4.8 
Tabulation of Individual Participant Data ............................................................ 32 
8.4.9 
Exploratory Analyses ............................................................................................ 32 
9 
SUPPORTING DOCUMENTATION AND OPERATIONAL CONSIDERATIONS ...................................... 33 
9.1 
Regulatory, Ethical, and Study Oversight Considerations .................................................. 33 
9.1.1 
Informed Consent Process ................................................................................... 33 
9.1.2 
Confidentiality and Privacy .................................................................................. 33 
9.1.3 
Future Use of Stored Specimens and Data .......................................................... 34 
9.1.4 
Key Roles and Study Governance ........................................................................ 34 
9.1.5 
Quality Assurance and Quality Control ................................................................ 35 
9.1.6 
Data Handling and Record Keeping ..................................................................... 35 
9.1.7 
Protocol Deviations .............................................................................................. 36 
9.1.8 
Publication and Data Sharing Policy .................................................................... 36 
9.1.9 
Conflict of Interest Policy ..................................................................................... 37 
9.2 
Additional Considerations .................................................................................................. 37 
9.3 
Protocol Amendment History ............................................................................................. 37 
10 
REFERENCES..................................................................................................................................... 38 
 
List of Exhibits 
1. 
Hybrid Type 1 Effectiveness-Implementation Design ......................................................................... 3 
2. 
Hybrid Type 1 Flow Diagram ............................................................................................................... 5 
3. 
Schedule of Activities .......................................................................................................................... 6 
4. 
SFP+FA Conceptual Model ................................................................................................................ 10 
5. 
Study Objectives, Endpoints, and Justifications ................................................................................ 15 
6. 
Study research Questions, Associated Hypotheses and outcomes, by evaluation Phase ................ 17 
7. 
Hybrid Type 1 Effectiveness-Implementation Design ....................................................................... 25 
8. 
Illustration of Difference-in-Differences Evaluation Model .............................................................. 27 
9. 
Overview of the Mixed-Methods Analysis for the Study’s Hybrid Type 1 Design ............................ 31 
10. 
Protocol Amendment History ........................................................................................................... 38 
 
 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
1 
STATEMENT OF COMPLIANCE 
The trial will be conducted in accordance with International Council on Harmonisation Good Clinical 
Practice (ICH GCP), applicable United States (U.S.) Code of Federal Regulations (CFR), and the National 
Center for Injury Prevention and Control (NCIPC) Terms and Conditions of Award. The Principal 
Investigator (PI) will assure that no deviation from or changes to the protocol will take place without prior 
agreement from the funding agency and documented approval from the Institutional Review Board 1, and 
the Investigational New Drug (IND) or Investigational Device Exemption (IDE) sponsor, if applicable, except 
where necessary to eliminate an immediate hazard(s) to the trial participants. All personnel involved in 
the conduct of this study have completed Human Subjects Protection and ICH GCP Training. 
The protocol, informed consent form(s), recruitment materials, and all participant materials will be 
submitted to the Institutional Review Board (IRB) for review and approval. Approval of both the protocol 
and the consent form(s) must be obtained before any participant is consented. Any amendment to the 
protocol will require review and approval by the IRB before the changes are implemented to the study. 
All changes to the consent form(s) will be IRB-approved; a determination will be made regarding whether 
new consent needs to be obtained from participants who provided consent, using a previously approved 
consent form. 
1 
PROTOCOL SUMMARY 
1.1 
SYNOPSIS  
 
Title: 
Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program: A Hybrid Type 1 
Effectiveness-Implementation Trial 
Grant Number: 
1 U01CE003482-01-00 
Study Description: 
Early exposure to Adverse Childhood Experiences (ACEs), such as parental substance use, increases the 
likelihood of future substance use and drug overdose, resulting in an intergenerational cycle of 
substance-related ACEs that can continue indefinitely if left uninterrupted. Community-level 
interventions may moderate the relationship between ACEs and substance use by providing an array of 
family support services and treatments to reduce disparities and improve reach and service linkages in 
the community. Although research suggests that effectively decreasing the prevalence and impact of 
ACEs and substance use requires addressing both family- and community-level factors in tandem, there 
is a critical gap within the evidence base pertaining to interventions that effectively integrate the two 
factors to prevent substance use and ACEs. RTI International and its established partners, the New 
Jersey Prevention Network and RWJBarnabas Health, will evaluate an intervention integrating New 
Jersey’s established, evidence-based Strengthening Families Program (SFP) with clinically trained, 
trauma-informed Family Advocates (FAs) who will assist families in accessing community resources. 
Specifically, this study will use a Hybrid Type 1 effectiveness-implementation design across 32 New 
Jersey communities experiencing a disproportionate burden of substance use and ACEs. 
Objectives*: 
 
Primary Objective: Conduct rigorous evaluations of prevention approaches implemented within 
communities that incorporate efforts to mitigate the harms of ACEs exposure and prevent future ACEs, 
while aiming to prevent substance use and overdose.  
The study has three aims: (1) use a cluster randomized controlled trial to test effectiveness of the 
SFP+FA intervention on substance use, overdose, and ACEs in 10 communities compared with SFP-only 
in 10 communities; (2) conduct a robust process evaluation informed by the Consolidated Framework 
for Implementation Research (CFIR) to explore implementation barriers and facilitators; and (3) conduct 
a cost evaluation to accurately estimate the costs required to implement SFP and SFP+FA and assess the 
cost-effectiveness of SFP+FA relative to SFP alone. Findings will provide a roadmap about the best ways 
to help disproportionately affected communities prevent substance use, overdose, and ACEs. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
2 
Endpoints*: 
Participants are considered to have completed the study after reaching the primary and secondary 
endpoints for this study: 
• Primary endpoint: the completion of the SFP7-17 curriculum, which consists of 11 sessions 
conducted over 10 to 14 weeks (depending upon family risk factors). 
• Secondary endpoint: the completion of a 6-month post-SFP completion survey used to assess the 
outcomes of participating families.  
These endpoints are further depicted in the Schedule of Activities, Section 1.3. 
Study Population: 
The target population for this study is families eligible for SFP7-17 (i.e., families with one or more 
caregivers and one or more children aged 7–17 years; with children 10-14 as the target age of children) 
that reside within 36 New Jersey communities experiencing high rates of ACEs and substance use and 
overdose.  
Phase* or Stage: 
This study uses a Hybrid Type 1 Effectiveness-Implementation Design,2 which is used to test a clinical 
intervention while gathering information on its delivery during the effectiveness trial or on its potential 
for implementation in a real-world situation. 
Description of 
Sites/Facilities 
Enrolling 
Participants: 
Participating families in the treatment and control conditions will be recruited to SFP7-17 via the 
established approach used by NJPN and its partners, which utilizes a combination of recruitment 
champions located in Family Success Centers, schools, and other community organizations. SFP7-17 
meetings will be held in each of the 36 participating New Jersey communities, with in-person meetings 
conducted at community organizations, churches, and other local sites.  
Families in the treatment condition will interact with a Robert Wood Johnson Behavioral Health, 
Institute for Prevention and Recovery (RWJBH IFPR) FA. The FAs will attend the first SFP7-17 session in 
person and then engage with families via phone calls or emails conducted from RWJBH IFPR offices. This 
study does not intend to include sites outside of the United States.   
Description of 
Study 
Intervention/Experi
mental 
Manipulation: 
Families in the treatment and control groups will participate in the SFP7-17 Group Class Curriculum for 
families with children ages 7-17. Parents and children participate in SFP7-17, both separately and 
together, as the curriculum has lessons for parents, teens, and children plus a joint Family Practice class. 
SFP7-17 meetings are 2 hours in length and are typically held in person (but families can participate 
remotely, during extenuating circumstances) and participating families complete 11 sessions over a 10- 
to 14-week period.  
Families in the treatment condition will be connected to clinically trained, trauma-informed FAs that 
will assess and refer families to services in the community. The intervention will provide wraparound 
supports to prevent ACEs and substance use and, critically, enable providers and community-based 
partners to align their services and resources in a way that addresses the community-level health needs 
of children and families. The FA component of the intervention will run concurrent to the SFP7-17 
sessions, with the FAs interacting weekly with families over the 10- to 14-week intervention period. On 
a weekly basis, FAs will conduct post-SFP-session check-ins with partners and caregivers, with 1-hour 
dedicated to each family. This 1-hour period will consist of a 20-minute phone call with families to 
discuss their needs, with the remaining 40 minutes used to debrief, make service referrals, and 
complete documentation.  
Study Duration*: 
Estimated 36 months, beginning October 1, 2023, and concluding by September 30, 2026. 
Participant 
Duration: 
It is estimated that it will take 9.5 to 10.5 months for participants to complete all study-related tasks. 
These tasks consist of the following: 
• 1 week for participating families to complete study consent forms and family intake forms 
• 10 to 14 weeks for families to complete the SFP7-17 curriculum.  
Participating families will complete a 6-month follow-up survey at approximately 6 months after 
completing the SFP7-17 curriculum.  
 
1.2 
SCHEMA  
This study’s Hybrid Type 1 effectiveness-implementation Design2 combines process and outcome 
evaluation components with a cost evaluation component to evaluate the intervention’s impact on family 
outcomes over a 3-year period (2023–2025). Hybrid Type 1 designs are ideally situated for rigorous 
evaluations where there is a strong evidence base for an intervention (such as SFP) and supporting 
evidence that the intervention would support applicability to new delivery methods (e.g., FAs) and new 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
3 
settings and populations (e.g., communities with high rates of overdose and ACEs). Exhibit 1 illustrates 
how each intervention phase fits together for the study. 
Phase 1 of the Hybrid Type 1 design consists of a process evaluation focused on pre-trial implementation, 
with the goal of understanding the complex implementation elements that influence effectiveness. During 
this phase, a combination of semi-structured interviews and focus groups will be used to understand 
potential barriers to accessing services, opportunities for community engagement and building 
relationships with community service providers, adaptations to SFP, and challenges to implementing the 
intervention with fidelity.  
Exhibit 1. Hybrid Type 1 Effectiveness-Implementation Design 
 
Phase 1 is followed by an outcome evaluation consisting of an effectiveness trial that employs a cluster 
randomized controlled trial (RCT) where 36 New Jersey communities are randomized to the treatment 
group consisting of SFP+FA or a control group consisting of SFP. Cluster RCTs offer a rigorous design for 
evaluating community-level interventions and provide key benefits in the form of mitigating spillover 
effects, reducing attrition, and increasing compliance from program staff and participants.3 This phase will 
use a combination of baseline, completion, and 6-month follow-up assessments to examine youth and 
parent/caregiver substance use and ACEs among 360 families within the 36 communities randomized to 
treatment and control. Other components of this phase will focus on monitoring fidelity, collecting cost 
data for the treatment and control groups, and conducting focus groups and semi-structured interviews 
to understand barriers and challenges to participating and the value-added by the FA component. 
The Post-Trial Implementation Phase will consist of a process evaluation focused on obtaining a deeper 
understanding of the potential barriers and facilitators to implementing the interventions, the challenges 
associated with delivering the intervention during the effectiveness trial, and potential implementation 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
4 
strategies that could maximize implementation. This evaluation phase will use a combination of semi-
structured interviews and focus groups with prevention agency program leads and program staff to 
understand implementation barriers and facilitators along with a survey of community service providers 
to obtain a system-level perspective on implementing an ACE-informed FA with SFP to prevent substance 
use and overdose.  
The cost evaluation will focus on the costs of implementing SFP+FA and SFP alone, the cost-effectiveness 
of SFP+FA compared with SFP alone, and the associated costs of scaling SFP+FA to serve additional 
families. The cost evaluation will use a combination of semi-structured interviews and an associated 
costing instrument with prevention agency program leads and brief surveys with program staff to identify 
the activities and costs associated with implementing SFP+FA. Data obtained from the prevention agency 
program lead interviews and program staff surveys will inform start-up cost estimations, implementation 
cost estimations, and cost-effectiveness analyses. 
Exhibit 2 provides a flow diagram for the study’s Hybrid Type 1 design. 
 
 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
5 
Exhibit 2. Hybrid Type 1 Flow Diagram 
 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
6 
1.3 
SCHEDULE OF ACTIVITIES  
Exhibit 3 provides the schedule of data collection activities. Activities are organized around each study 
component. The schedule is organized around key timepoints in the study, including pre-screening, 
various SFP 7-17 sessions and graduation, and 6 months after SFP graduation.     
Exhibit 3. Schedule of Activities 
Instrument (Respondent) 
Pre-screening  
(Pre-consent) 
Pre-Implementation 
SFP 7-17 Sessions 1-2 
SFP 7-17 Session 3 
SFP 7-17 Sessions 4-9 
SFP 7-17 Session 10 & 
Graduation (Primary 
Endpoint) 
6-months post-SFP 
Graduation 
(Secondary Endpoint) 
SFP Eligibility Determination (Recruiters) 
⚫ 
 
 
 
 
 
 
Informed Consent (Families) 
 
⚫ 
 
 
 
 
 
Pre-Trial Implementation  
Community Service Provider Baseline Surveys (SFP Family 
Coaches, FAs, Community Service Providers) 
 
⚫ 
 
 
 
 
 
Pre-implementation Interviews (FAs) 
 
⚫ 
 
 
 
 
 
Pre-implementation Focus Groups (SFP Family Coaches) 
 
⚫ 
 
 
 
 
 
Effectiveness Trial 
Fidelity Survey (SFP Family Coaches) 
 
 
⚫ 
⚫ 
⚫ 
⚫ 
 
Fidelity Survey (Families) 
 
 
⚫ 
⚫ 
⚫ 
⚫ 
 
SFP Pre-test Survey & ACEs/Social Needs Screener (Families) 
 
 
 
⚫ 
 
 
 
Weekly Family Referral Summary (SFP Family Coaches and 
FAs) 
 
 
⚫ 
⚫ 
⚫ 
⚫ 
 
Epic EHR Data Collection (FA) 
 
 
⚫ 
⚫ 
⚫ 
⚫ 
 
SFP Post-test Survey (Families) 
 
 
 
 
 
⚫ 
 
Post-Trial Implementation 
Community Service Provider Follow-up Survey (SFP Family 
Coaches, FAs, Community Service Providers) 
⚫ 
 
SFP 6-month Follow-up Survey & ACEs/Social Needs 
Screener (Families) 
⚫ 
Post-implementation Interviews (Program Leads) 
⚫ 
 
Post-implementation Interviews (FAs) 
⚫ 
 
Post-implementation Focus Groups (SFP Family Coaches) 
⚫ 
 
Cost Evaluation 
Costing Instrument and Interview (Program Leads) 
 
 
⚫ 
 
Per Program Implementation Cost Survey 1 (SFP Coach) 
 
 
 
 
 
⚫ 
 
Monthly Implementation Cost Survey 2 (Family Advocate) 
 
 
 
 
⚫ 
⚫ 
 
 
 
 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
7 
2 
INTRODUCTION 
2.1 
STUDY RATIONALE  
Early exposure to ACEs, such as parental substance use, increases the likelihood of future substance use 
and drug overdose, resulting in an intergenerational cycle of substance-related ACEs that can continue 
indefinitely if left uninterrupted. Parental substance use is among the most reported ACE in the United 
States with one in eight children living with a parent who uses alcohol or drugs resulting in “significant 
impairment.” When left untreated, parental substance use can create adverse childhood environments4 
that threaten household safety, stability, and bonding and heighten the child’s own likelihood of 
substance use and drug overdose. The primary goals and objectives of this study is to evaluate the 
effectiveness of a novel intervention intended to mitigate the impact of existing ACEs and prevent future 
ACEs on substance use and overdose among vulnerable families.  
Interventions, such as the use of patient navigators, FAs, and family navigation programs that connect 
parents experiencing substance use to treatment and an array of family support services have been 
demonstrated as increasing protective factors against ACEs and reducing disparities while improving reach 
and creating service linkages in the community. Critically, these interventions can have a systems-level 
impact that benefits the larger community through better coordination and distribution of limited 
resources.4 Although research suggests that effectively decreasing the prevalence and impact of ACEs and 
substance use requires addressing both family- and community-level factors in tandem,4 there is a critical 
gap within the evidence base pertaining to interventions that effectively integrate approaches to prevent 
substance use and ACEs. 
To address this critical gap in the evidence base, RTI International (RTI) and its established partners, the 
New Jersey Prevention Network (NJPN) and the RWJBarnabas Health Institute for Prevention and 
Recovery (RWJBH IFPR), will integrate a novel intervention that combines family- and community-level 
factors within 36 New Jersey communities experiencing a disproportionate burden of substance use and 
ACEs. This intervention will integrate clinically trained, trauma-informed FAs into New Jersey’s established 
evidence-based SFP. FAs will assist families in accessing community resources for substance use and ACE 
prevention and treatment. This study will test whether this integrated intervention increases referrals and 
linkages to community resources, increases positive family functioning, and reduces youth and parent 
substance use and ACEs compared with families participating in SFP alone. The study will also test whether 
integrating a combined family- and community-level intervention produces systems-level change by 
aligning services and resources and subsequently decreasing the prevalence of substance use and ACEs at 
the community level. 
The proposed study will address the gap in the evidence base by rigorously evaluating a novel intervention 
that effectively integrates family- and community-level factors to prevent substance use, overdose, and 
ACEs. The study’s research design will build the evidence base for community-level interventions by using 
a Hybrid Type 1 design that integrates outcome, process, and cost evaluations to optimize the likelihood 
that the intervention strategies shown to be effective will be sustainable when implemented in future 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
8 
communities. The findings from this study will provide a roadmap for how disproportionately affected 
communities can implement SFP+FA to prevent substance use, overdose, and ACEs. 
ACEs and Substance Use in New Jersey. Prevalence of ACEs and Substance Use. Prior to the COVID-19 
pandemic, New Jersey’s state government estimated that over 40% of children (approximately 782,000) 
experienced at least one ACE,5 slightly above the national average of 30%. The percentage of children in 
New Jersey who have reportedly experienced two or more ACEs has ranged between 12%–16% between 
2016 to 2019, also aligning with national estimates of children experiencing multiple ACEs.6 Among the 
state’s children under 5 years old, 33% are reported to have experienced one or more ACE.5 As observed 
nationally, exposure to ACEs disproportionately affects minority children compared with non-Hispanic, 
White children in New Jersey. Among African American/Black children, the rates (between 26%–29%) are 
triple the size when compared with rates reported among non-Hispanic white children (between 8%–13%) 
over the same period. The percentage of Hispanic/Latinx children in New Jersey with more than two ACEs 
is also higher (between 14%–18%) than rates of non-Hispanic white children. 
In 2020, New Jersey’s Department of Human Services reported 82,254 treatment admissions where the 
primary diagnosis was substance abuse.7 This robust number is in line with the heightened landscape of 
substance use/misuse across the country during the height of the COVID-19 pandemic.8 Examining 2020 
statistics in New Jersey, alcohol was found to be the primary substance of use in over 30% of the 
admissions while heroin was identified in over 40% of admissions. One-quarter of all admissions were 
among individuals in the 35–44 age group. Additionally, a reported 3,046 individuals died of a drug 
overdose in New Jersey in 2020,9 which was a slight uptick from 2019. Before the onset of the COVID-19 
pandemic, drug overdose death rates were declining among non-Hispanic White individuals (40.3 per 
100,000 in 2018 and 36.2 per 100,000 in 2019) while rates were increasing among non-Hispanic Black 
individuals (36.3 per 100,000 in 2018 and 39.9 per 100,000 in 2019) and slightly increasing among Hispanic 
individuals (18.4 per 100,000 in 2018 and 19.4 per 100,000 in 2019).10 Much like the rest of the country, 
overall substance use in New Jersey has since increased.11 
Strengthening Families Program. The SFP7-17 Curriculum is a new 11-session class curriculum from the 
developers of the original SFP. The curriculum consists of an evidence-based family skills training program 
intended to reduce behavior problems and substance use by improving parenting skills, parent–child 
relationships, and family communication. Parents, youth, and children attend weekly group sessions 
lasting 2 hours for 11 weeks with each 2-hour session comprising individual parent and child practice 
sessions and a combined family session that allows the parents and children to practice what they learned 
in the individual sessions. New Jersey has established a widespread SFP offering through partnership 
between the New Jersey Department of Human Services’ Health Division of Family Development and Child 
Care Resource Referral agencies in each county. SFP has been recognized by the Centers for Disease 
Control and Prevention (CDC)12 for its effectiveness in reducing known risk factors for ACEs13 and 
decreasing youth substance use, including opioid misuse.14-16 
Indeed, SFP is a well-documented evidence-based intervention that has been tested in numerous states 
and countries and adapted for diverse populations and settings.17-24 A recent scoping review25 of SFP found 
positive effects on drug abuse prevention and protective parenting factors in the United States, with 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
9 
primary outcomes that include delayed use of alcohol and other drugs, decreased exposure to substance 
use, prevention of new users, long-term decrease of drug abuse, long-term academic success, and 
increased school engagement. However, this same review underscored a lack of research on program 
adoption, reach, and maintenance. In particular, the authors emphasized a need to consider and integrate 
contextual aspects (e.g., geographic, socioeconomic) into the implementation process, as they directly 
impact program effectiveness and reach.26 Indeed, the SFP facilitators are often not trained in making 
effective referrals, which necessarily limits the degree to which SFP can effectively identify and address 
issues related to substance use or make service linkages in the community. 
Patient Navigators Programs. The patient navigator model is a versatile approach to improving access to 
resources and reducing disparities across diverse settings.27 In general, this model uses clinically trained 
staff to assist patients, families, and caregivers in navigating health care systems. To date, New Jersey has 
implemented a range of patient navigator programming aimed at connecting individuals to needed 
resources. For example, the New Jersey Department of Children and Families has established a navigator 
program specifically aimed at facilitating access to substance use services, with the expectation that staff 
are fully knowledgeable of the continuum of care available in the community. 
Although most of the existing literature on patient navigators has focused on facilitating access to medical 
care (e.g., for populations with chronic disease),28 models have also been studied in behavioral health 
settings and among underserved populations. In a study of people living with HIV who had substance use 
disorders (SUDs), participants who worked with a patient navigator showed higher rates of being linked 
to care than those who did not.29; 30 Moreover, the patient navigator was found to mitigate medical 
mistrust, proving especially beneficial for populations with higher rates of medical mistrust—often 
populations who have been historically abused by the healthcare industry.30 Similar models, such as peer 
advocates, have also shown success in improving patient satisfaction with mental health services.31 Family 
care navigation and FA programs have been demonstrated to increase client engagement and cooperation 
between family support services.1 In sum, patient navigator models can significantly aid screening, service 
referrals, and adherence, making them an effective strategy to improve reach and create service linkages 
in the community. 
Proposed Integration of SFP and FA Model. Research suggests that to effectively decrease the prevalence 
and subsequent impact of ACEs, including substance use, it is not enough to target only family- or 
community-level factors—rather, interventions should consider both contexts in tandem.32 The strategic 
coordination of supports and services in and across the community is a critical element in disrupting the 
relationship between ACEs and substance use. New Jersey has recently demonstrated its readiness for 
such efforts via the NJ ACEs Action Plan, which outlines the state’s commitment to becoming a trauma-
informed, healing-centered state with a focus on child and family resiliency. Notably, there remains a clear 
need for community-based supports and cross-sector collaboration in all settings in New Jersey to provide 
care to all residents, including those in communities experiencing disproportionate burden. 
The proposed study combines the respective strengths of the SFP and FA models to address the need for 
ACE-informed prevention of substance use and overdose. Specifically, we will augment New Jersey’s 
established SFP with the addition of clinically trained, trauma-informed FAs to assess and refer families to 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
10 
services in the community. The intervention will provide wraparound supports to prevent ACEs and 
substance use and, critically, enable providers and community-based partners to align their services and 
resources in a way that addresses the community-level health needs of children and families. 
Conceptual Model and Theory of 
Change. The study will investigate the 
effectiveness of the SFP+FA interven-
tion as a systems-level strategy to 
affect immediate change in families 
and the surrounding community and 
long-term 
changes 
in 
ACEs 
and 
substance use. The conceptual model 
and associated theory of change are 
driven 
by 
early 
concepts 
of 
Bronfenbrenner’s theory,33 with special 
consideration paid to short-term, 
intermediate, and long-term outcomes 
across ecological systems (see Exhibit 
4). This approach will allow the 
research team to assess family- and 
community-level measures and the 
mechanisms 
through 
which 
the 
proposed intervention works. 
Research has demonstrated that SFP is 
effective 
in 
delivering 
short-term 
outcomes, including increased family 
functioning, ACEs awareness, and parent self-efficacy and decreased youth’s intentions to use.17-24 
However, SFP is limited in its reach to community-level systems and outcomes. The goal of the FA model 
intervention is to further that reach and subsequently bring about improvements in intermediate and 
long-term outcomes. The hypothesized mechanism of change is two-fold: first, families in the SFP+FA 
intervention will receive an increased number of service referrals, tailored based on needs flagged in the 
screening tools administered by their assigned FAs. These referrals, in conjunction with ongoing SFP 
programming, will result in increased adherence and service utilization than those receiving SFP alone. 
Second, these service referrals and repeated contacts in between systems and agencies will facilitate 
increased systems linkages, thus breaking down the silos currently present in New Jersey and providing 
equitable access to treatment services across all communities. Together, these community-level process 
changes will contribute to the reduction in ACEs, substance use, and drug overdoses captured in long-
term outcomes. 
Exhibit 4. SFP+FA Conceptual Model 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
11 
2.2 
BACKGROUND  
Adverse Childhood Experiences (ACEs). ACEs are preventable, potentially traumatic events that occur in 
childhood but carry lasting, negative impacts on individual and community health outcomes. Although 
some of these effects are not detected until adulthood,34 others—like increased risk of substance use—
are observed much earlier in the lifespan.12; 35 As a result, early exposure to ACEs can result in cascading 
effects on substance use and related adverse outcomes over time, like drug overdose. Left unaddressed, 
ACEs and their associated outcomes cycle across generations and permeate into the community.36 The 
complex interplay between ACEs and substance use thus holds significant relevance to U.S. drug overdose 
prevention efforts. 
An estimated one-half to two-thirds of U.S. adults have experienced at least one ACE, with up to one-
quarter reporting three or more.4; 12; 37-40 Level of exposure serves as a critical indicator of subsequent risk, 
because a greater number of ACEs directly correlates with a greater incidence and severity of negative 
health and well-being outcomes: studies have found that people with six or more ACEs die nearly 20 years 
earlier than their peers without ACEs. The relationship between ACEs and substance use is similarly 
observed in a dose-response fashion: for every unit increase in ACEs, the chances of developing alcohol, 
cocaine, or opioid dependence are nearly doubled. Research has suggested that ACEs account for up to 
two-thirds of all cases of SUD.41 
ACEs are associated with a number of adverse outcomes, spanning domains of physical34; 42 and mental12; 
43; 44 health, and intra-38; 41 and interpersonal44-46 behaviors. The effects of ACEs on some outcomes, such 
as heart disease or diabetes, are not detected until adulthood. In contrast, ACEs play a more immediate 
role in substance use initiation among youth. Adolescents exposed to five or more ACEs are 15 times more 
likely to report recent (i.e., past 30 days) opioid misuse than those with no ACEs. In total, over 70% of 
recent opioid misuse among adolescents is attributable to ACEs.47 The estimated risk of non-medical 
prescription drug use among adolescents increases by 62% for every additional ACE.48 Findings from 
several studies suggest that earlier initiation of substance use is linked to subsequent development of 
dependence49-51 or progression to more illicit, lethal drugs and forms of use.52 
In addition to the early and cumulative effects of ACEs on substance use, there is an intergenerational 
component that poses key considerations for efforts to prevent and mitigate harm. Namely, parental 
substance use is among the most commonly reported ACEs in the United States;53 one in eight children 
lives with a parent who uses alcohol or drugs resulting in “significant impairment.” This, as with all ACEs, 
threatens household safety, stability, and bonding and heightens the child’s own likelihood of substance 
use and drug overdose. The cyclical nature of the relationship between ACEs and substance use is both a 
contributor to and beneficiary of the ongoing opioid epidemic. In 2020, over 90,000 people in the United 
States died of a drug overdose.54 Although rates were highest among individuals in the 35–44 age group 
(54.9 per 100,000), the largest percent increase in drug overdose deaths from 2019 to 2020 occurred 
among the 15–24 age group (49% increase). 
Community-Level Factors for ACEs. Community-level factors play a substantial role in increasing risk of or 
protecting against ACEs within families.55 Community characteristics and social determinants of health 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
12 
(SDOH) such as low socioeconomic status,56 lack of neighborhood services,57 and low collective efficacy58 
all have negative ramifications on family functioning and, accordingly, have been linked to a number of 
ACEs related to parenting, abuse, and mental illness.59-62 The relationship between community context 
and family functioning is carried out by numerous, interwoven threads. Indeed, poverty is often a proxy 
indicator for other risk factors of relevance for ACE prevention efforts: people in poor communities tend 
to have fewer economic and educational opportunities and safe housing options, are more likely to 
experience food insecurity, and have reduced access to resources such as community activities, health 
care services, and childcare.63-65 Racial inequities, caused by systemic racism and oppression, place 
communities and families of color at heightened risk of poverty and its associated challenges; poverty 
rates among African American, American Indian, and Hispanic children were two to three times higher 
(31%, 30%, 23%, respectively) than those of white children (10%) in 2019. Whole communities can be 
susceptible to developing a socially isolating environment in which ACEs can proliferate, resulting in a 
cycle of family- and community-level trauma.4; 64 
The relationship between community-level factors and ACEs is made all the more complex with the 
introduction of substance use. To demonstrate, prior research has shown that either high parent exposure 
to ACEs or high alcohol vendor density in the community is sufficient to increase risk of ACEs in 
adolescents.61 When considering substance use as an outcome, similar considerations apply. Given that 
substance use is highly correlated with ACEs, it remains vulnerable to many of the same risk factors (e.g., 
access to drugs and alcohol, social isolation).55 However, community-level factors may moderate this 
association in key respects. 
The positive relationship between child maltreatment report rates and drug overdose mortality, for 
instance, is stronger in non-metro and rural communities than in metro areas,66 suggesting that decreased 
access to mental and physical health care and substance use treatment in urban areas may contribute to 
drug overdose outcomes. Other research findings have pointed to community-level factors of social 
exclusion and relative deprivation in explaining evidence of substance use outcomes clustering by 
geographic area.67 
In promising contrast, community-level characteristics can serve as protective factors against ACEs and 
their subsequent impacts.55 For example, increased community cohesion—namely, instilling support and 
trust among residents—has been shown to mediate the risk of child maltreatment39 and moderate 
intergenerational continuity in ACEs.61 Community treatments and supports can also fend off social 
isolation, a known risk factor for substance use, which has been robustly evidenced by the recent COVID-
19 pandemic–associated lockdowns.68 Efforts to address ACEs and substance use outcomes can be 
enhanced and bolstered by attending to these community-level factors, in conjunction with any targeted 
individual- or family-level interventions. 
2.3 
RISK/BENEFIT ASSESSMENT  
Study participants may experience minimal risks throughout the course of the study. Known potential 
risks may include the following:  

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
13 
• Psychological Harms. Participants may experience stress and feelings of embarrassment or guilt 
related to participating in the intervention and answering survey questions pertaining to 
substance use and ACEs.  
• Social and Economic Harms. Because of the nature of the intervention and the associated data 
collection activities, participants could experience social or economic harms. Substance use and 
ACEs are particularly sensitive topics, and the study could yield information about study 
participants that could be considered stigmatizing.  
• Privacy and Breaches of Confidentiality Risks. Losses of privacy or breaches of confidentiality 
could result in additional embarrassment, or in more extreme circumstances, loss of 
employment or criminal prosecution.    
A discussion of the strategies that will be employed to minimize risks is included in Section 2.3, 
Assessment of Potential Risks and Benefits.  
2.3.1 Known Potential Benefits  
Study participants may experience a combination of direct and indirect potential benefits throughout the 
course of the study. Known potential benefits may include the following: 
• Direct Benefits 
− 
SFP is a well-documented evidence-based intervention that has been tested in numerous 
states and countries and adapted for diverse populations and settings.17-24 A recent scoping 
review25 of SFP found positive effects on drug abuse prevention and protective parenting 
factors in the United States, with primary outcomes that include delayed use of alcohol and 
other drugs, decreased exposure to substance use, prevention of new users, long-term 
decrease of drug abuse, long-term academic success, and increased school engagement. 
− 
Research has demonstrated that SFP is effective in delivering short-term outcomes, 
including increased family functioning, ACEs awareness, and parent self-efficacy and 
decreased youth intentions to use. 17-24 
− 
SFP has been recognized by the CDC12 for its effectiveness in reducing known risk factors for 
ACEs13 and decreasing youth substance use, including opioid misuse.14-16 
− 
FAs: Research has shown that patient navigator programs, upon which the FA component of 
the intervention is based, provide a versatile approach to improving access to resources and 
reducing disparities across diverse settings.27 
− 
Although most of the existing literature on FAs and patient navigators has focused on 
facilitating access to medical care (e.g., for populations with chronic disease),27 models have 
also been studied in behavioral health settings and among underserved populations. In a 
study of people living with HIV who had SUDs, participants who worked with a patient 
navigator showed higher rates of being linked to care than those who did not.1; 29-31 
− 
These models have also been found to mitigate medical mistrust, proving especially 
beneficial for populations with higher rates of medical mistrust—often populations who 
have been historically abused by the healthcare industry.30 Similar models, such as peer 
advocates, have also shown success in improving patient satisfaction with mental health 
services.31 
• Indirect Benefits 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
14 
− 
This study is anticipated to offer societal/aspirational benefits through the evaluation of a 
community-level intervention, with the study’s findings on the effectiveness of the 
intervention anticipated to provide benefits to society and future patients.  
− 
More specifically, this study incorporates an increased focus on identifying shared risk 
factors69 and systems dynamics70 that represent an important shift toward sustaining 
outcomes associated with community-level prevention programming.  
− 
Community-level prevention should mirror the focus on ACEs based on a fundamental 
understanding of how cumulative risk factors increase negative outcomes but also by 
understanding how environmental risk factors interact to explain the trajectory and severity 
of substance use and overdoses within a community. And although an effective approach 
must first address individual and environmental ACEs, it is equally important to identify the 
most effective delivery strategies and mitigate the outcomes associated with them. As such, 
the proposed strategy provides a mechanism to address both the individual and family 
factors (via SFP) that are associated with substance use and overdose while ensuring proper 
access to the resources (via FAs) necessary to improve outcomes.  
− 
Moreover, FAs will facilitate capacity building and communication across the systems and 
agencies that support families as they address substance use and potential overdose. It is 
this innovation that is believed will have an additional lasting and sustainable community-
level impact compared with other similar interventions.  
2.3.2 Assessment of Potential Risks and Benefits  
The known potential risks and benefits described in the preceding sections demonstrate that the potential 
risk of harm to study participants are outweighed by the potential direct and indirect benefits. Participants 
may encounter minimal risks throughout the course of the study, including psychological harms, social 
and economic harms, and privacy and breaches of confidentiality risks.  
In addition, this study will incorporate the following established strategies for minimizing risks: 
• The research conducted under the proposed study will be conducted in compliance with the 
U.S. Department of Health and Human Services (DHHS) Policy for Protection of Human Research 
Subjects (45 CFR part 46), the standards of RTI’s Office of Research Protection, and in 
accordance with the Certificate of Confidentiality requirement for CDC-supported research.  
• The study will provide prospective study participants with an informed consent form that 
provides a clear understanding of the known potential risks. 
• The study will record data without identifiers. 
• The study will collect the minimum data necessary for the research. 
• The study will perform only procedures that are necessary to achieve the study objectives.  
• The study will securely store data in accordance with best practices: 
− 
Data will be stored in an RTI project share within RTI’s Health Insurance Portability and 
Accountability Act (HIPAA)-compliant network. The network is a segregated network for 
securing sensitive data as the loss of confidentiality, integrity, or availability of the data are 
expected to have a serious adverse effect on individuals. The network conforms to HIPAA 
requirements and NIST 800-53 security controls, including separation from other RTI 
network services and requiring user training and two-factor authentication. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
15 
− 
We expect all study data to be electronic. However, should a need arise to print individual-
level data, RTI will store those files in a locked cabinet. 
− 
The study team will access project shares on the HIPAA-compliant network using their RTI-
issued laptops. Desktop security is achieved via Microsoft’s Windows Server operating 
system security features (i.e., user identification and password are required for access, 
lockout of account upon repeated entry of an invalid password). 
The aforementioned risks to subjects are reasonable in relation to the anticipated benefits to subjects and 
the importance of the knowledge that may be expected to result from this study. Furthermore, this study 
will incorporate a number of established processes and strategies for minimizing risks and protecting 
human subjects. Finally, the combination of direct and indirect benefits is expected to considerably 
outweigh the known risks. All study participants, regardless of assignment condition, will participate in 
SFP, an evidence-based intervention shown to be effective in decreasing substance use, reducing risk 
factors for ACEs, and increasing parent self-efficacy and family functioning. Study participants assigned to 
the treatment condition will receive additional direct benefits in the form of improved access to resources, 
greater linkages to care, reduced rates of medical mistrust, and increased levels of satisfaction. Finally, 
indirect benefits are anticipated in the form of studying a novel community-level intervention designed 
to provide lasting and sustainable community-level impacts in the form of increased capacity and 
communications across the systems and increased agencies supporting families experiencing ACEs and 
substance use issues. The findings from this study are anticipated to provide longer term 
societal/aspirational benefits in the form of enhanced knowledge that is likely to benefit future 
communities and individuals who experience disproportionate levels of ACEs and substance use issues.  
3 
OBJECTIVES AND ENDPOINTS  
Exhibit 5 describes the study objectives and endpoints, along with a justification for endpoint selection. 
Additional information is included in Section 8.4.2, Analysis of the Primary Endpoint(s) and Section 8.4.3, 
Analysis of the Secondary Endpoint(s).   
Exhibit 5. Study Objectives, Endpoints, and Justifications 
Objectives 
Endpoints 
Justification for Endpoints 
Primary Objectives 
To use a cluster RCT to test 
the effectiveness of the 
SFP+FA intervention on 
substance use, overdose, and 
ACEs in 18 communities 
compared with SFP-only in 
18 communities.  
 
 
Primary Endpoints 
• SFP Post-test Survey 
(completed by families at 
SFP7-17 graduation) 
• Weekly Family Referral 
Summaries (completed by 
SFP Family Coaches and 
FAs throughout 
intervention until SFP7-17 
Graduation) 
• Community Service 
Provider Follow-up Survey 
(completed by SFP Family 
Coaches, FAs, Community 
Service Providers following 
SFP7-17 Graduation) 
Secondary Endpoints 
Primary Endpoints 
• The SFP Post-Test Survey will collect key outcome 
data for measuring the intervention’s impact on 
substance use, perceptions of harm, and risk  
• The Weekly Family Referral Summaries will collect 
key outcome data for measuring the intervention’s 
impact on referrals to SUD services and non-
clinical community service providers 
• The Community Service Provider Follow-up Survey 
will collect key outcome data for measuring the 
intervention’s long-term impact on community-
level change via change in system linkages 
Secondary Endpoints 
• The ACEs/Social Needs Screener will collect key 
outcome data for measuring the intervention’s 
impact on the prevalence of ACEs and social needs 
among participating families 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
16 
Objectives 
Endpoints 
Justification for Endpoints 
• ACEs/Social Needs 
Screener (completed by 
families at Session 3 of 
SFP7-17 and 6 months 
after SFP7-17 Graduation) 
• SFP 6-Month Follow-up 
Survey (completed by 
families 6 months after 
SFP7-17 Graduation) 
• The SFP 6-Month Follow-up Survey will collect key 
outcome data for measuring the intervention’s 
long-term impact on substance use, perceptions of 
harm, and risk  
Secondary Objectives 
To conduct a robust process 
evaluation informed by the 
CFIR to explore 
implementation barriers and 
facilitators 
To conduct a cost evaluation 
to accurately estimate the 
costs required to implement 
SFP and SFP+FA and assess 
the cost-effectiveness of 
SFP+FA relative to SFP alone. 
 
Primary Endpoints (Process 
Evaluation) 
• Post-Trial Focus groups 
(completed with SFP and 
FA delivery staff following 
SFP7-17 Graduation) 
• Post-Trial Program Lead 
Interviews (completed 
with Prevention Agency 
Program Leads following 
SFP7-17 Graduation) 
Primary Endpoints (Cost 
Evaluation) 
• Costing Instrument and 
Interview (completed by 
Prevention Agency 
Program Leads following 
SFP7-17 Graduation) 
• Per Program 
Implementation Cost 
Survey (completed by SFP 
Family Coaches following 
SFP7-17 Graduation) 
• Monthly Implementation 
Cost Survey (completed by 
FAs monthly)  
Primary Endpoints (Process Evaluation) 
• The Post-Trial Focus Groups and Prevention 
Agency Program Lead Interviews will collect key 
process evaluation data related to (a) observed 
challenges and associated solutions that 
participating agencies encountered when 
integrating the FA component into SFP; (b) 
observed barriers and facilitators families 
encountered when accessing services; (c) how the 
implementation of SFP+FA impacts family access 
to services, cross-system coordination, and 
communication 
Primary Endpoints (Cost Evaluation) 
• The Costing Instrument and Interview, Per 
Program Implementation Cost Survey, and 
Monthly Implementation Cost Survey will collect 
key cost evaluation data will inform start-up cost 
estimations, implementation cost estimations, and 
cost-effectiveness analyses 
 
4 
STUDY DESIGN 
4.1 
OVERALL DESIGN 
This section builds upon the content provided in Sections 1.1 Synopsis and 1.2 Schema, by providing 
additional detail on the study design. The table below (Exhibit 6) provides an overview of the study 
research questions, associated hypotheses, and outcomes by evaluation phase.  
Stage of the trial. This study uses a Hybrid Type 1 Effectiveness-Implementation Design2 which is used to 
test a clinical intervention while gathering information on its delivery during the effectiveness trial or on 
its potential for implementation in a real-world situation. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
17 
Exhibit 6. Study research Questions, Associated Hypotheses and outcomes, by evaluation Phase 
Evaluation 
Phase 
Research Question 
Associated Hypotheses 
Outcomes (instrument) 
Effectiveness 
Trial 
What is the effectiveness of SFP+FA, 
in contrast to SFP alone, on 
substance use, perceptions of harm 
and risk?  
Families in SFP+FA communities will 
have a decreased prevalence of 
substance use, increased 
perceptions of harm, and decreased 
risk at the end of the intervention 
period than families in SFP-only 
communities 
Substance use, perceptions of 
harm, and risk (SFP Pre-test, Post-
test, and 6-month follow-up 
surveys) 
What is the effectiveness of SFP+FA, 
in contrast to SFP alone, on the 
prevalence of ACEs and social 
needs? 
Families in SFP+FA communities will 
have a decreased prevalence of 
ACEs and social needs than families 
in SFP-only communities 
ACEs (ACEs/Social Needs Screener 
[Baseline], ACEs/Social Needs 
Screener [6-month follow-up]) 
What is the effectiveness of SFP+FA, 
in contrast to SFP alone, on referrals 
to services?  
Referrals to service will be higher at 
communities where SFP+FA is 
implemented than SFP-only 
communities 
Referral to service rates (Weekly 
Family Referral Summaries; 
Monthly Epic referral to services 
summary report and 6-month 
follow-up surveys) 
What is the effectiveness of SFP+FA, 
in contrast to SFP alone, on service 
utilization?  
Service utilization rates will be 
higher in communities where 
SFP+FA is implemented than SFP-
only communities 
Service utilization rates (Monthly 
Epic referral to services summary 
report for clinical services referred 
and 6-month follow-up surveys) 
What is the effectiveness of SFP+FA, 
in contrast to SFP alone, on 
treatment/service adherence?  
Treatment/service adherence will 
be higher in communities where 
SFP+FA is implemented than SFP-
only communities 
Service adherence rates (6-month 
follow-up surveys) 
What is the effectiveness of SFP+FA, 
in contrast to SFP alone, on referrals 
to non-clinical services?  
Referrals to non-clinical services will 
be higher in communities where 
SFP+FA is implemented than SFP-
only communities 
Referral to non-clinical service rates 
(Weekly Family Referral Summaries; 
Monthly Epic referral to services 
summary report, and 6-month 
follow-up surveys) 
What is the effectiveness of SFP+FA, 
in contrast to SFP alone, on 
establishing system linkages?  
Communities implementing SFP+FA 
will have greater changes in clinical 
and non-clinical referral networks 
(such as changes in network size, 
edges, density, centrality, and 
reciprocity) than communities 
implementing SFP alone 
System linkages (i.e., network size, 
edges, density, centrality, 
reciprocity) (Community Service 
Provider Follow-up Survey) 
Phase 2: 
Post-
Implementat
ion Trial 
How, if at all, does the 
implementation of SFP facilitate 
increased cross-system coordination 
and communication among the 
agencies that support families as 
they address substance use and 
potential overdose? 
Communities implementing SFP+FA 
will report increased cross-system 
coordination and communication 
among the agencies that support 
families as they address substance 
use and potential overdose relative 
to communities implementing SFP 
alone 
Lessons learned 
Social connectedness across 
systems (Post-Trial Focus Groups 
and Program Leads Interviews) 
Community service providers who 
have received referrals from SFP+FA 
will report some knowledge 
of/familiarity with SFP and will 
indicate SFP+FA role builds capacity 
and enhances communication 
across systems 
Provider knowledge on SFP; Level of 
effectiveness/value of FA; 
Provider's role in facilitating safe 
environment; Success building trust 
with families (Post-Trial Focus 
Groups and Program Lead 
Interviews; Community Service 
Provider Follow-up Survey) 
Cost 
Evaluation 
What are the costs to implement 
SFP+FA and SFP alone? 
The average cost of delivering 
SFP+FA is greater than the average 
cost of delivering SFP alone 
Average cost 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
18 
Evaluation 
Phase 
Research Question 
Associated Hypotheses 
Outcomes (instrument) 
What is the cost-effectiveness of 
SFP+FA relative to SFP alone? 
The cost-effectiveness ratio in 
terms of cost per linkage is below 
commonly accepted willingness to 
pay thresholds (e.g., $50,000 USD) 
Cost per linkage 
 
Design of the trial. The effectiveness component for this study’s Hybrid Type 1 Effectiveness-
Implementation Design uses a cluster RCT, where 36 New Jersey Communities are randomized to the 
treatment group consisting of SFP+FA or a control group consisting of SFP-only.   
Community selection and randomization. In accordance with this study’s focus on evaluating the 
effectiveness of a community-level intervention within disproportionately affected communities, this 
study will target 36 New Jersey communities over a 3-year period experiencing high rates of ACEs and 
substance use and overdose. These 36 communities will be initially drawn from the populations across 
approximately 200 communities where NJPN and RWJBH IFPR operate SFP and FA. Under this approach, 
communities were first ranked in terms of three ACEs and opioid-related vulnerability metrics71-74  
(overdose-focused SDOH, opioid overdose-involved hospitalizations, and child maltreatment case rates) 
and assigned into quadrants based on average rankings across the three metrics. NJPN then solicited 
program study interest through a survey form, prioritizing communities in the most vulnerable quadrant 
(four) and second most vulnerable quadrant (three) with regards to outreach. The survey gauged county 
provider and community interest, past (and recent) experience with SFP as well as community buy-in level 
and readiness to implement SFP in the upcoming year (or in future years) and any additional communities 
that should be considered for the intervention. Communities that responded to the surveys were then 
compiled into a list to be randomized.  
Specification of the method for assigning participants to study groups. The study will employ a pairwise 
random sampling process to randomly assign the 36 communities and the families they serve to treatment 
and control conditions. Pairwise randomization is a special form of stratified randomization where pairs 
of units (i.e., communities) are matched on important characteristics and the members of each pair are 
then randomly assigned to either the treatment or control conditions. This randomization approach is 
particularly effective for cluster RCTs and has been shown to be effective in achieving a balance between 
treatment and control groups and increasing statistical power.  
4.2 
SCIENTIFIC RATIONALE FOR STUDY DESIGN 
The primary goals and objectives of the study are to mitigate the impact of existing ACEs and prevent 
future ACEs of substance use and overdose among vulnerable families. Interventions, such as patient 
navigators, FAs, and family navigation programs that connect parents experiencing substance use to 
treatment and an array of family support services have been demonstrated as increasing protective 
factors against ACEs and reducing disparities while improving reach and creating service linkages in the 
community. Critically, these interventions can have systems-level impact that benefits the larger 
community through better coordination and distribution of limited resources.4 Although research 
suggests that effectively decreasing the prevalence and impact of ACEs and substance use requires 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
19 
addressing both family- and community-level factors in tandem,4 there is a critical gap within the evidence 
base pertaining to interventions that effectively integrate approaches to prevent substance use and ACEs. 
To address this critical gap in the evidence base, this study will evaluate a novel intervention that 
combines family- and community-level factors within 36 New Jersey communities experiencing a 
disproportionate burden of substance use and ACEs. This intervention will integrate clinically trained, 
trauma-informed FAs into New Jersey’s established, evidence-based SFP. FAs will assist families in 
accessing community resources for substance use and ACE prevention and treatment.  
The proposed study will address the gap in the evidence base by rigorously evaluating a novel intervention 
that effectively integrates family- and community-level factors to prevent substance use, overdose, and 
ACEs. The study’s research design will build the evidence base for community-level interventions by using 
a Hybrid Type 1 design that integrates outcome, process, and cost evaluations to optimize the likelihood 
that the intervention strategies shown to be effective will be sustainable when implemented in future 
communities. The findings from this study will provide a roadmap for how disproportionately affected 
communities can implement SFP+FA to prevent substance use, overdose, and ACEs. 
4.3 
JUSTIFICATION FOR INTERVENTION 
The intervention will combine the SFP with an FA model to address the need for ACEs-informed prevention 
of substance use in New Jersey. FAs will provide a trauma-informed and person- and family-centered 
approach to parent education, screening and assessment, resource navigation, and supportive follow-up 
for families impacted by ACEs. FAs will be integrated into the SFP to build trusted relationships with 
families and assist with the identification of resources to address or prevent negative impacts on the social 
care needs of the family. 
SFP will be conducted in person over the course of 11 2-hour sessions. The minimum acceptable 
participation to the intervention is at least eight SFP sessions. The FA will attend the initial SFP session and 
graduation and will provide supportive follow-up at a minimum of once a week through virtual platforms 
or phone calls. Many families struggle to navigate the care system that serves individuals impacted by 
trauma and ACEs. The FA will provide supportive and individualized care and wraparound supports that 
meets the specific needs of each family. The role of the FA is to build a trusting relationship with families 
while increasing access to and navigation of resources or services and decreasing barriers and stigma 
experienced by individuals and families. 
SFP has been implemented in over 35 countries and is culturally sensitive rather than culturally specific 
and has been successfully adapted for African American, Asian/Pacific Islander, Hispanic, and American 
Indian families and has been translated into Spanish, Portuguese, Russian, Dutch, Swedish, Norwegian, 
German, Austrian, Slovenian, Italian, French, Thai, and Chinese. All staff working on the program will be 
familiar with the target population’s cultures and languages and will be able to administer screenings and 
data collection tools to patients with limited English proficiency14 using My Accessible Real-Time Trusted 
Interpreter (MARTTI) devices, a service currently used by RWJBH, which is in accordance with the 
culturally and linguistically appropriate services standards for ensuring cultural and linguistically 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
20 
appropriate services are made available. This service is provided at no cost to families and is facilitated to 
provide timely access to all community and health care and services. Additionally, RWJBH includes cultural 
competency and Ending Racism Together training as part of orientation and annual training to ensure that 
participants and patients receive equitable care. Upon program initiation, staff will be required to 
complete training on program logistics and cultural competency. 
4.4 
END-OF-STUDY DEFINITION 
A participant is considered to have completed the study if they have completed the baseline assessment, 
at least eight intervention sessions, the SFP post-test survey (primary endpoint), and the 6-month follow-
up survey (secondary endpoint), as shown in the Schedule of Activities, Section 1.3. 
5 
STUDY POPULATION 
5.1 
INCLUSION CRITERIA 
To be eligible to participate in this study, a family must meet all of the following criteria: 
• Either reside in, or attend SFP meetings, within one of the 36 New Jersey communities with 
disproportionate levels of ACEs and SUD issues that are assigned to either the treatment or 
control conditions via the study’s cluster RCT design 
• Meet family eligibility requirements: 
− 
One or more adult caregivers 
− 
One or more adolescents, ages 7 to 17 
• Caregiver provision of signed and dated informed consent form  
• For children, informed assent and parental informed consent to participate in the study 
• Stated willingness to comply with all study procedures and lifestyle considerations (see Section 
5.3, Lifestyle Considerations) and availability for the duration of the study  
• Willingness to adhere to the regimens of the SFP and FA interventions 
• Access to necessary resources for participating in a technology-based intervention (i.e., 
computer, smartphone, internet access) 
5.2 
EXCLUSION CRITERIA 
A family who meets any of the following criteria will be excluded from participation in this study: 
• Caregiver has previously completed SFP with one or more children, ages 7 to 17  
• Intellectual disabilities (i.e., cognitive impairments that would prohibit the completion of the SFP 
curriculum or data collection instruments) 
• Language difficulties (caregivers and children must read and understand spoken English) 
5.3 
LIFESTYLE CONSIDERATIONS 
N/A. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
21 
5.4 
SCREEN FAILURES 
Screen failures are defined as individuals who consent to participate in this study, but it is subsequently 
determined that these individuals do not meet one of more of the study inclusion criteria or who meet 
one or more of the exclusion criteria. Because of the study’s cluster RCT design (i.e., communities rather 
than individuals are randomly assigned) and the nature of the intervention (i.e., individuals who meet 
inclusion criteria are part of a cohort who are randomly assigned to either the treatment of control 
conditions), screening failures will be excluded from the study and re-screening of participants will not 
occur.  
5.5 
STRATEGIES FOR RECRUITMENT AND RETENTION 
The study will leverage NJPN’s established recruiting and retainment processes. The recruitment process 
involves a combination of universal promotion to all families, targeted outreach to specific high-risk 
families, and incentives for recruiting other families to participate in SFP. Prevention agencies in the 
participating communities will use local champions in schools and community groups (e.g., the YMCA, 
Boys and Girls Clubs, and faith-based organizations) to recruit families to participate in the study. Copies 
of recruitment materials are submitted with the protocol. Retention efforts involve a combination of 
supportive services (e.g., transportation, childcare) to reduce barriers to participation, along with an array 
of regular incentives throughout the program and financial incentives for perfect attendance and 
completing the program. 
In addition to the existing incentives offered to families for participating in SFP 7-17, this study will offer 
a pair of incentives to participating families and community service providers. Participating 
parents/caregivers will receive a $50 gift card incentive for completing the Baseline ACEs/Social Needs 
Screener during the third SFP 7-17 session and an additional $50 gift card for completing the 6-Month 
Follow-up survey and ACEs/Social Needs Screener. Up to 15 community service providers in each of the 
36 participating communities will be offered $20 gift cards for completing the Pre-Implementation 
Community Service Provider Survey and $25 gift cards for completing the Post-Implementation 
Community Service Provider Survey.     
6 
STUDY INTERVENTION(S) OR EXPERIMENTAL MANIPULATION(S) 
6.1 
STUDY INTERVENTION(S) OR EXPERIMENTAL MANIPULATION(S) ADMINISTRATION 
6.1.1 Study Intervention or Experimental Manipulation Description 
The effectiveness trial component of the evaluation consists of a cluster RCT where 36 New Jersey 
communities experiencing high rates of ACEs and substance use and overdose are randomized to the 
treatment group consisting of SFP+FA or a control group consisting of SFP. Treatment communities will 
administer the SFP7-17 curriculum, which consists of 11 sessions conducted over 10 to 14 weeks, to 
eligible families and will integrate the FA component, where clinically trained, trauma-informed FAs will 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
22 
assist families in accessing community resources. Control communities will operate under a business-as-
usual condition where they administer the SFP7-17 curriculum to eligible families.   
Families in the treatment and control communities will have completed the study after reaching the 
study’s primary and secondary endpoints. The primary endpoint is the completion of the SFP7-17 
curriculum, which consists of 11 sessions conducted over 10 to 14 weeks (depending upon family risk 
factors). The secondary endpoint is the completion of a 6-month post-SFP completion survey used to 
assess the outcomes of participating families. These endpoints are further depicted in the Schedule of 
Activities, Section 1. 
6.1.2 Administration and/or Dosing 
Families in the treatment and control communities will participate in the SFP7-17 curriculum, which 
consists of 11 sessions. SFP sessions are 2 hours in length and are typically held in person, though families 
can participate remotely, when necessary. A full-dose consists of attending all 11 SFP sessions, while the 
minimum acceptable dosage is attending at least 8 SFP sessions. Further details on the use of an 
interventionist, measures to minimize bias, and study intervention/experimental manipulation adherence 
are discussed in the following sections.  
6.2 
FIDELITY 
6.2.1 Interventionist Training and Tracking 
The process evaluation research team will observe at least one SFP training held for participating agencies 
prior to implementation, in part to ensure consistency with SFP guidelines and to identify key metrics to 
monitor for variability. SFP Family Coaches will complete a form at the end of each session to indicate 
their own adherence to session delivery and any unique circumstances that would impact consistent 
administration of the study intervention. In addition, participants will complete a fidelity monitoring form 
at the end of each session to report the family coach’s knowledge, empathy, presence, and preparedness. 
The study team will track these findings on an ongoing basis.  
6.3 
MEASURES TO MINIMIZE BIAS: RANDOMIZATION AND BLINDING 
Randomization of communities will be performed as described in Section 4.1. Implementation staff will 
not be blinded to study condition because of logistical constraints of program delivery and the active 
participation of FAs in the SFP+FA arm of the study. Enrolled families will be aware of the presence of the 
FA in the SFP+FA treatment arm but will not be told of an SFP-only condition. Bias or contamination 
because of contact with other study staff or condition is not likely due to randomization at the community 
level. 
6.4 
STUDY INTERVENTION/EXPERIMENTAL MANIPULATION ADHERENCE 
At the end of each session, implementers will complete a fidelity monitoring form to track attendance of 
each family, including parental and youth units, and key measures of participant engagement. Attendance 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
23 
will be tracked over time for each family to ensure adequate dosage of the control/experiment conditions. 
Participants will need to complete at least eight sessions to remain an active participant. 
6.5 
CONCOMITANT THERAPY 
N/A. 
6.5.1 Rescue Therapy 
N/A. 
7 
STUDY INTERVENTION/EXPERIMENTAL MANIPULATION DISCONTINUATION AND 
PARTICIPANT DISCONTINUATION/WITHDRAWAL 
7.1 
DISCONTINUATION OF STUDY INTERVENTION/EXPERIMENTAL MANIPULATION 
When a subject discontinues from SFP or the FA component of the intervention but not from the study, 
remaining study procedures will be completed as indicated by the study protocol. If a clinically significant 
finding is identified (including changes from baseline) after enrollment, the investigator or qualified 
designee will determine if any change in participant management is needed. Any new clinically relevant 
finding will be reported as an adverse event 50. 
The data to be collected at the time of study intervention discontinuation will include the reason(s) for 
discontinuing the participant from the intervention, and methods for determining the need to discontinue 
7.2 
PARTICIPANT DISCONTINUATION/WITHDRAWAL FROM THE STUDY 
Participants are free to withdraw from participation in the study at any time upon request. 
An investigator may discontinue a participant from the study for the following reasons: 
• Significant study intervention non-compliance (e.g., non-compliance with the SFP, absence from 
more than three SFP sessions, or if a participant presents as a threat to themselves or others). 
• Lost to follow-up; unable to contact subject (see Section 7.3, Lost to Follow-Up) 
• Any event or medical condition or situation occurs such that continued collection of follow-up  
study data would not be in the best interest of the participant or might require an additional 
treatment that would confound the interpretation of the study 
• The participant meets an exclusion criterion (either newly developed or not previously 
recognized) that precludes further study participation 
The reason for participant discontinuation or withdrawal from the study will be recorded in participant 
records.  
7.3 
LOST TO FOLLOW-UP 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
24 
A participant will be considered lost to follow-up if they fail to attend eight or more SFP sessions. The 
following actions will be taken: 
• The site will attempt to contact the participant, identify opportunities for making up the missed 
sessions, counsel the participant on the importance of attending scheduled SFP sessions, and 
ascertain if the participant wishes to or should continue in the study 
• Before a participant is deemed lost to follow-up, the investigator or designee will make every 
effort to regain contact with the participant (where possible, three telephone calls or emails). 
These contact attempts will be documented in the participant’s study records.  
• If the participant continues to be unreachable, they will be considered to have withdrawn from 
the study with a primary reason of lost to follow-up. 
8 
STATISTICAL CONSIDERATIONS  
8.1 
STATISTICAL HYPOTHESES 
Included below are the hypotheses associated with the primary and secondary endpoints for the 
effectiveness trial.  
Primary Endpoints  
• Families in SFP+FA communities will have a decreased prevalence of substance use, increased 
perceptions of harm, and decreased risk at the end of the intervention period than families in 
SFP-only communities. 
• Referrals to service will be higher in communities where SFP+FA is implemented than SFP-only 
communities.  
• Service utilization rates will be higher in communities where SFP+FA is implemented than SFP-
only communities. 
• Treatment/service adherence will be higher in communities where SFP+FA is implemented than 
SFP-only communities.  
• Referrals to non-clinical services will be higher in communities where SFP+FA is implemented 
than SFP-only communities. 
• Communities implementing SFP+FA will have greater changes in clinical and non-clinical referral 
networks (such as changes in network size, edges, density, centrality, and reciprocity) than 
communities implementing SFP alone. 
Secondary Endpoints 
• Communities implementing SFP+FA will have greater changes in clinical and non-clinical referral 
networks (such as changes in network size, edges, density, centrality, and reciprocity) than 
communities implementing SFP alone. 
• Families in SFP+FA communities will have a decreased prevalence of ACEs and social needs than 
families in SFP-only communities. 
8.2 
SAMPLE SIZE DETERMINATION 
Sample size for the outcomes study was based on power analyses for the primary analysis model to be 
used with the primary endpoints of the study (e.g., substance use, perceptions of harm, and risk, 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
25 
treatment referral and utilization). This model estimates differences in changes pre- to post-intervention 
for the SFP and SFP plus FA groups (a differences in differences model). Power was estimated as the 
minimal detectable effect size (MDES) in Cohen d metric and used a Type I error rate of 0.05. Optimal 
Design 3 was used to estimate power for this design. 
In a cluster RCT such this, the MDES for a repeated-measure model depends on the number of clusters 
(36 communities in this study), the average number of participants per cluster (10 families completing per 
site), the number of assessments, and the intraclass correlation (ICC; the extent to which families within 
clusters are more like one another than families from other clusters). ICC is not known a priori for this 
population and outcomes and likely varies across outcomes. However, previous studies of the extent of 
clustering at the community level suggest ICCs will be low, in the range of 0.01 to 0.0575,76 
MDES was plotted as a function of the known parameters with ICC varying on the X axis and is shown in 
the exhibit below. Curves are shown for both the average anticipated completed number of families per 
site (10) and the anticipated enrolled families per site (15), and 36 communities evenly randomized to SFP 
and SFP+FA. For low values of ICC, 0.01 to .05, the MDES is approximately 0.35 or below.  
Sensitivity to attrition was examined by estimating the anticipated number of enrolled families in each 
site (15) and comparing the power to curve of enrolled vs completes. The expected attrition rate has 
relatively little impact on power, with a difference of about .05 at very low ICC and about .025 at ICC = 
0.05. 
Exhibit 7. Hybrid Type 1 Effectiveness-Implementation Design 
MDES by ICC 
 
 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
26 
 
8.3 
POPULATIONS FOR ANALYSES 
Analyses of primary and secondary endpoints will use all families in each study arm that complete SFP7-
17. Completion will be defined as those that complete eight or more sessions. Families that miss three or 
more SFP sessions will be considered as lost to follow-up and excluded from evaluation analysis models. 
All retained families will be analyzed according to their randomization (i.e., following intention to treat). 
Sensitivity analyses will predict drop-out/attrition as a function of baseline characteristics of families using 
baseline data from the SFP pre-test survey and initial Weekly Family Referral Summaries. 
8.4 
STATISTICAL ANALYSES 
8.4.1 General Approach 
Qualitative Analysis of Process Evaluation Data 
Qualitative data will be analyzed using a hybrid deductive–inductive analytic approach that emphasizes 
addressing key constructs of the CFIR, such as intervention characteristics, outer and inner settings, 
characteristics of individuals, and process while allowing for emerging patterns. Interview and focus group 
transcripts will be uploaded to the qualitative data analysis software, ATLAS.ti. Transcripts will be coded 
by individual team members using a codebook guided by CFIR domains and the constructs within them 
while allowing team members to identify emerging themes in the data. We will check reliability by having 
two coders code a subset of transcripts and comparing the level of agreement. A Cohen’s Kappa statistic 
will be calculated to check for coder consistency in the application of the codebook. If a high level of 
consensus is not reached, team members will work to resolve conflicts until reaching a level of high 
intercoder agreement (0.80). 
Through an iterative process, the qualitative study team will continually create, refine, and discard codes. 
The qualitative study team will hold bi-weekly data meetings during the analysis phase to discuss coding. 
As transcripts are discussed and coded, we will also write analytical memos to record observations, refine 
preliminary themes, and capture emerging themes. When all transcripts have been coded and discussed, 
we will engage in an iterative process of searching data using ATLAS.ti, comparing data segments from 
searches, and writing analytic memos to answer the process evaluation’s research questions, thereby 
developing a full understanding of the barriers and facilitators to implementing FAs within SFP. 
Quantitative Analysis of Effectiveness Trial Data 
All inferential statistics will use a Type I error rate of 0.05 and two-tailed tests.  
Descriptive Statistics. Preliminary analyses will be used to characterize and explore the data. Means, 
frequencies, and correlations will describe the distributions of demographics, key covariates, and the 
outcome variables. Simple bivariate regression models will be used to examine the groups for baseline 
differences. Variables will be assessed for outliers and other distributional violations. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
27 
Program Effects. The difference in family outcomes from exposure to SFP+FA vs. SFP-only will be examined 
with generalized linear mixed models (GLMM)—that is, hierarchical linear models or multilevel regression 
models.75-80 These models can accommodate the repeated assessments at baseline, immediate post-
treatment, and 6-month follow-up. GLMM can simultaneously account for nesting of families in 
communities to account for any non-heterogeneity attributable to location. The evaluation model will 
consist of a three-timepoint latent growth model (LGM).77; 81 This model examines how family outcomes 
change over time and incorporates random effects (variance components) for each family’s initial status 
(intercept) and rate of change or slope.79 The LGM model will test how the SFP and SFP+FA groups change 
from baseline through the 6-month follow-up and the difference in those two rates of change. The 
difference in rates of change (sometimes called a difference-in-differences model)76; 80 will be the primary 
evaluation outcome. 
Exhibit 8 shows a difference-in-difference model in which Panel A shows no intervention effect (the 
difference A2 A1 is the same as the difference B2 B1), and Panel B demonstrates an intervention effect 
(A2 A1 is not equal to B2 B1).  
The LGM approach provides more options compared with ANOVA or other traditional mean comparison 
techniques by being able to incorporate covariates into an analysis. Covariates may be at any level 
included in the GLMM, such as the individual, family, or community levels. All outcomes assessed will be 
examined with these models and will begin with random effects at the family and community levels for 
both the intercept and slope, and the covariance between intercepts and slopes at each level will be 
included. Models may be iteratively adjusted to remove random effects and covariances as necessary, 
because it is common for not all outcomes of interest to require all sources of variability. 
Exhibit 8. Illustration of Difference-in-Differences Evaluation Model 
 
 
Missing Data Strategies. The impact of missing data will be minimized through the use of likelihood-based 
estimators,81-88 which are available for multilevel models of the proposed difference-in-differences model 
for the evaluation. These likelihood methods yield unbiased estimates and maximally efficient standard 
errors without sacrificing cases (and thus maximizing statistical power). The proposed method of 
accommodating missing data is fully appropriate when data are missing truly at random (missing 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
28 
completely at random [MCAR]) or predicted by other variables (such as baseline scores and covariates) in 
a given model but independent of the potential values of the outcome itself (i.e., missing at random 
[MAR]). The MAR and MCAR mechanisms are termed ignorable missing data84 because they can be 
rendered unproblematic for analysis with the proper estimation technique and model (e.g., including the 
variable that predicts missingness as a predictor of the outcome). Although there is no general test of the 
MAR assumption, MAR-appropriate estimators are typically maximally efficient and unbiased even when 
the data are not truly MAR or MCAR (i.e., the missing data are non-ignorable).87 However, if the pattern 
of missing data is suspected to be non-ignorable, then the research team will incorporate auxiliary 
variables in the analysis model105 or pattern-mixture models that are adept at obtaining estimates for 
longitudinal studies with attrition caused by levels of outcomes may be used.85 
Network Analysis of Effectiveness Trial Data 
The findings from the community service provider survey will be used to examine the intervention’s 
effectiveness in achieving community-level change via changes in the referrals and linkages between FAs, 
SFP Family Coaches, and community service providers in each of the 36 communities participating in the 
effectiveness trial. Surveys will be administered pre- and post-implementation of the intervention within 
each of the 36 communities, with individuals in the three respondent categories asked a series of 
questions about the organizations they refer or collaborate with for any ACEs, substance use, or mental 
health services. Respondents (referred to as egos within the community service provider surveys) will be 
asked to list up to 10 of the main organizations that they refer or collaborate with followed by a second 
set of questions asking about referrals and collaborations with selected community service providers.89; 90 
Answers to these questions will provide a matrix of interactions among FAs, SFP Family Coaches, and 
community service providers that will facilitate the construction of egocentric networks within each of 
the 36 communities for the periods prior to and after implementation of the intervention. 
This study will use a multi-step process to analyze the networks for the treatment and control 
communities. In the first step, network diagrams will be constructed using the R software package. 
Descriptive network analyses will be conducted on each of the networks to examine key network 
measures for the communities in the treatment and control conditions. These analyses will move from 
the simple to more complex network measures:91-93  
• Network size: the number of nodes (i.e., FAs, SFP Family Coaches, and community service 
providers) in a given network.  
• Edges: connections (directed and undirected) among nodes within a network. 
• Density: how close a network is to being fully connected, ranging from no connections at all to 
all possible connections among nodes are made. 
• Centrality: the degree to which a network is organized around a central node by comparing the 
proportion of edges to a central node to all other edges in a network. 
• Reciprocity: the fraction of all possible connections in which nodes are mutually connected. 
• Triadic closure: the addition of an edge that closes a 2-path (i.e., three nodes connected by two 
edges) to form a triangle (or among three nodes, A, B, and C, connections A-B and A-C exist, and 
a new connection B-C is formed).   

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
29 
• Assortativity: the extent that nodes in a network are associated with other nodes in the 
network, being of similar sort or being of opposing sort. 
In the next step of the analysis, mixed-effects models will be estimated to examine the intervention’s 
effectiveness in achieving community-level change via overall and configurational changes in the referrals 
and linkages between FAs, SFP Family Coaches, and community service providers in treatment and control 
communities. The random intercept and random coefficient mixed-effects models will incorporate a two-
level structure with repeated measurements (i.e., pre- and post-implementation) nested within 
communities.94 The model will incorporate random slopes for communities and random intercepts for 
time. The following two-level model will estimate the impacts of the intervention on referrals and linkages 
between FAs, SFP Family Coaches, and community service providers: 
𝑦𝑖𝑗= 𝛽0 + 𝜇0𝑗+ 𝛽1𝑇𝑟𝑒𝑎𝑡1 𝑖𝑗+ 𝛽2𝑋2 𝑖𝑗+ ⋯+ 𝛽𝑝𝑋𝑝 𝑖𝑗+ 𝜇1𝑋1𝑖𝑗+ 𝜀𝑖𝑗 
where 𝑦𝑖𝑗 refers to the outcome for timepoint 𝑖 in community 𝑗; 𝛽0 is the overall mean of the outcome 
across all groups; 𝜇0𝑗represents the random intercept; 𝛽1𝑇𝑟𝑒𝑎𝑡1 𝑖𝑗 is an indicator equal to one for 
communities assigned to the treatment group and zero for those assigned to the control group; 𝛽2𝑋2 𝑖𝑗+
⋯+ 𝛽𝑝𝑋𝑝 𝑖𝑗 represents a vector of covariates; 𝜇1𝑋1𝑖𝑗 represents the random coefficient; and 𝜀𝑖𝑗 is an 
individual-level error term. 
The final step of the analysis will consist of formulating Exponential Random Graph Models95 (ERGMs), a 
class of statistical models for social networks, to examine the structure of networks in the treatment and 
control communities post-implementation. ERGMs account for the presence and absence of networks ties 
(i.e., edges) and provide an approach to modeling overall network structure by modeling small local tie-
based structures, such as reciprocity and triangles. These models permit inferences about whether a given 
network has significantly more or less of a given feature (such as reciprocity or triadic closures) than one 
would expect and provide a deeper understanding of how and why network ties arise. This study will 
estimate a series of ERGMs to model network features in the treatment and control group communities95: 
𝑃𝜃 (𝐺) = 𝑐𝑒𝜃1𝑧1 (𝐺)+𝜃2𝑧2 (𝐺)+⋯+ 𝜃𝑝𝑧𝑝 (𝐺) 
where the probability of a given network G is given by a sum of network statistics (𝑧 in the equation above, 
which represent counts of the number of network configurations in the given network G) weighted by 
parameters (𝜃 in the equation above) inside an exponential (and where 𝑐 is a normalizing constant).  
Quantitative Analysis of Cost Evaluation Data 
The collected cost data will be used to calculate average program start-up costs and ongoing program 
implementation costs primarily from the funder perspective and from the societal perspective, reflecting 
the costs from the perspectives of all funders plus the value of any in-kind donations of time or other 
supplies or equipment. In cost analyses, costs will be calculated for the following resource categories: 
labor, materials/supplies, equipment, and other resource categories. Mean costs will be calculated by 
intervention assignment (SFP+FA or SFP-only), and cost variability will be examined by intervention, staff 
mix, or communities. The sensitivity of costs will also be examined against assumptions about unit costs 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
30 
or other input values. Cost estimates, particularly the costs of SFP, will be compared with other estimates 
in the research literature along with the primary drivers of differences or similarities in costs across 
implementations of SFP.96; 97 These costs will be reported as costs per site per year and costs per family 
per year. 
The total program costs for SFP+FA and SFP will be estimated and subsequently used to calculate the 
incremental cost of SFP+FA relative to SFP-only. These analyses will compare differences in start-up costs 
and ongoing program costs to implement the programs during the study period. Fixed costs, which do not 
vary based on the number of families served, will be compared with variable costs, which increase for 
each family or site added. Understanding the breakdown of costs between start-up and implementation 
is necessary for accurately estimating the costs of program scale-up or new program implementation in 
other settings. 
The study team will also explore the feasibility of conducting budget impact analysis. A budget impact 
analysis would first identify the organizations and agencies that incur costs and experience cost savings 
resulting from SFP+FA or SFP-only. These interventions may affect service utilization in multiple sectors, 
including the healthcare system, the criminal justice system, and the child welfare/protective services 
system. A budget impact analysis will consider a time horizon beyond the trial period, such as 5 or 10 
years, to analyze whether and how the intervention affects longer term outcomes in each of the various 
systems that impact families. One aspect of this feasibility study is to determine whether data are 
available to estimate the budget impact from the state, federal, or other funder perspectives. A potential 
barrier is limited access to appropriate data from all relevant systems to conduct the analysis. Although it 
is possible to assess impacts on emergency department and inpatient health care utilization using local 
data from Medicaid claims or hospital discharges, it may be more challenging to obtain data from non-
health social services systems. An important part of this study will be to assess the feasibility of obtaining 
relevant data from non-health social services systems. 
Mixed-Methods Analysis of Process, Effectiveness, and Cost Data 
In the final phase of the analysis, the evaluation data will be analyzed using a mixed-methods analysis 
approach where qualitative and quantitative data collected from the process, outcomes, and cost 
components of the Hybrid Type 1 design are systematically integrated98 and analyzed. Mixed-methods 
analysis helps answer questions that could not be answered by the qualitative or quantitative approaches 
alone, thereby enriching results, strengthening the overall reliability of a study’s findings, and providing 
the opportunity to uncover unique insights and findings that would have been otherwise neglected using 
a single method.99-101 Exhibit 9 provides an overview of the study’s mixed-methods analysis, which 
provides a systematic approach to integrating and interpreting data from the outcome evaluations’ 
effectiveness trial, the process evaluation’s focus on implementation barriers and facilitators, and the cost 
evaluation’s cost and cost-effectiveness estimates. 
The findings from the qualitative and quantitative analyses of process, outcome, and cost data will be 
systematically integrated using joint display tables that provide a visual framework for intentionally 
integrating findings with a clear rationale, thereby illuminating insights beyond separate quantitative and 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
31 
qualitative analyses.102 This mixed-methods analysis provides a critical benefit to the study by providing a 
robust understanding of the intervention’s effectiveness, implementation barriers and facilitators, and 
cost-effectiveness and thereby enhancing the likelihood that the intervention strategies shown to be 
effective will be sustainable when implemented in future communities. 
8.4.2 Analysis of the Primary Endpoint(s) 
Please refer to Section 3, Objectives and Endpoints and Section 9.4.1, Statistical Analyses: General 
Approach for further details on the steps associated with analyzing the primary endpoints.  
8.4.3 Analysis of the Secondary Endpoint(s) 
Please refer to Section 3, Objectives and Endpoints and Section 9.4.1, Statistical Analyses: General 
Approach for further details on the steps associated with analyzing the secondary endpoints.  
8.4.4 Safety Analyses 
N/A.  
Exhibit 9. Overview of the Mixed-Methods Analysis for the Study’s Hybrid Type 1 Design 
 
 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
32 
8.4.5 Baseline Descriptive Statistics 
Preliminary analyses will be used to characterize and explore the data. Means, frequencies, and 
correlations will describe the distributions of demographics, key covariates, and the primary and 
secondary endpoint outcome variables. Tables will display frequencies and counts for categorical items 
and means, standard errors, and minimums/maximums for continuous items. Variables will be assessed 
for outliers and other distributional violations. Violations/outliers will be handled on a case-by-case basis, 
with some options for handling these including setting the offending value to missing or winsorizing. 
Simple bivariate regression models with appropriate response distributions (e.g., binary/logistic) will be 
used to examine the intervention groups for baseline differences.  
In addition to simple descriptives statistics, we will generate comparable frequencies and means using 
multilevel models that incorporate the clustering of families within communities. In addition to cluster-
adjusted descriptives statistics, these models will be used to provide baseline estimates of intraclass 
correlations.101 The ICC estimates will indicate the extent to which variables of interest are more similar 
across families within communities (i.e., the extent to which there is clustering). 
8.4.6 Planned Interim Analyses  
N/A. 
8.4.7 Sub-Group Analyses 
Membership in subgroups based on family or community-level characteristics may impact the degree to 
which both SFP and SPF+FA change outcomes and how different the changes are across the two 
intervention groups. These subgroups based on demographics or risk factors will not be analyzed 
individually. To address the question of how these factors attenuate or enhance the program’s 
effectiveness, we will use moderation analysis using the program effects model described earlier. 
Moderation of program effects will be examined by adding the moderator (e.g., child, gender) and the 
interactions of the moderator by group (SFP vs. SFP+FA) by time to the LGM program effect models for 
primary and secondary endpoints. Moderation of efficacy will be found when the interaction of group, 
time, and moderator is significant, indicating that the difference between slopes (e.g., treatment vs. 
comparisons) is not the same at different levels of the moderating variable. When significant moderation 
is found, the effect will be further examined by estimation of simple slopes, or rates of change at specific 
values of the moderator variable. 
8.4.8 Tabulation of Individual Participant Data 
Individual participant data, including measures and time points will be de-identified and compiled within 
a combined dataset that will be used to facilitate the analyses described in the preceding sections. Please 
refer to Section 10.1.3, Confidentiality and Privacy for further details on the steps taken to securely store 
participant data and ensure confidentiality.  
8.4.9 Exploratory Analyses 
N/A. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
33 
9 
SUPPORTING DOCUMENTATION AND OPERATIONAL CONSIDERATIONS 
9.1 
REGULATORY, ETHICAL, AND STUDY OVERSIGHT CONSIDERATIONS 
9.1.1 Informed Consent Process 
Consent/assent and Other Informational Documents Provided to Participants 
Consent forms describing the study intervention, study procedures, and risks in detail will be given to 
participants, and written documentation of informed consent will be completed prior to starting the study 
intervention. The following consent materials are submitted with this protocol: parent/caregiver consent 
form (with parental permission for child’s participation in research) and youth assent form. 
Consent Procedures and Documentation 
Informed consent will be administered by local champions in schools and community groups (e.g., the 
YMCA, Boys and Girls Clubs, and faith-based organizations) who are responsible for recruiting families to 
participate in the SFP. The local champions will electronically administer the forms to parent/caregivers 
and youth using the REDCap platform. Parents/caregivers will sign and date the informed consent forms 
and provide parental permission for a focal youth (between the ages of 12 and 17) to participate in the 
study. The focal youth will complete the assent form by providing an online assent where they 
acknowledge that by continuing on to complete the associated survey, they agree to participate in the 
research study.  
The parent/caregiver consent and the youth assent forms will be available in English and Spanish and 
copies of the English and Spanish versions of the  parent/caregiver consent and the youth assent form are 
submitted with this protocol. In the event that speakers of a language other than English or Spanish need 
to complete the consent and assent forms, local champions and the study team will also have access to 
MARTTI devices, a service currently used by RWJBH, which is in accordance with the culturally and 
linguistically appropriate services standards for ensuring cultural and linguistically appropriate services 
are made available.  
9.1.2 Confidentiality and Privacy  
Participant confidentiality and privacy is strictly held in trust by the participating investigators, their staff, 
the safety and oversight monitor(s), and the sponsor(s) and funding agency. This confidentiality is 
extended to the data being collected as part of this study. Data that could be used to identify a specific 
study participant will be held in strict confidence within the research team. No personally identifiable 
information from the study will be released to any unauthorized third party without prior written approval 
of the sponsor/funding agency.  
All research activities will be conducted in as private a setting as possible. 
The study monitor, other authorized representatives of the sponsor or funding agency, representatives of 
the Institutional Review Board,1 regulatory agencies, or representatives from organizations supplying the 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
34 
product may inspect all documents and records required to be maintained by the investigators, including 
medical records (office, clinic, or hospital) for the participants in this study. The study sites will permit 
access to such records. 
The study participant’s contact information will be securely stored at each site for internal use during the 
study. At the end of the study, all records will continue to be kept in a secure location for as long a period 
as dictated by the reviewing IRB, Institutional policies, or sponsor/funding agency requirements. 
Study participant research data, which is for purposes of statistical analysis and scientific reporting, will 
be transmitted to and stored on an RTI project share within RTI’s HIPAA-compliant network. This will not 
include the participant’s contact or identifying information. Rather, individual participants and their 
research data will be identified by a unique study identification number. The study data entry and study 
management systems used by sites and by RTI research staff will be secured and password protected. At 
the end of the study, all study databases will be de-identified and archived in accordance with the NCIPC 
Data Management Plan. In accordance with the NCIPC Data Management Plan, the Co-Principal 
Investigators will ensure all mechanisms used to share data will include proper plans and safeguards for 
the protection of privacy, confidentiality, and security for data dissemination and reuse (e.g., all data will 
be thoroughly de-identified and will not be traceable to a specific study participant). Plans for archiving 
and preserving the data in the long-term will be implemented, as appropriate.  
Certificate of Confidentiality  
Section 301(d) of the Public Health Service (PHS) Act. Section 301(d), as amended by Section 2012 of the 
21st Century Cures Act, P.L. 114-255 (42 U.S.C. 241(d)), states that the Secretary, Health and Human 
Services (HHS), shall issue Certificates of Confidentiality (Certificates) to persons engaged in biomedical, 
behavioral, clinical, or other research activities in which identifiable, sensitive information is collected. In 
furtherance of this provision, CDC-supported research commenced or ongoing after December 13, 2016, 
in which identifiable, sensitive information is collected, as defined by Section 301(d), is deemed issued a 
Certificate and therefore required to protect the privacy of individuals who are subjects of such research. 
In accordance with CDC policy, Certificates issued in this manner will not be issued as a separate document 
but are issued by application of this term and condition to the federal award for this study. 
9.1.3 Future Use of Stored Specimens and Data  
Data collected for this study will be analyzed and stored at within RTI’s HIPAA-compliant network. After 
the study is completed, the de-identified, archived data will be transmitted to and stored at the CDC NCIPC 
data repository (in accordance with the NCIPC Data Management Plan), for use by other researchers, 
including those outside of the study. When the study is completed, access to study data or samples will 
be provided through the CDC NCIPC data repository. 
9.1.4 Key Roles and Study Governance 
Co-PI 
Co-PI 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
35 
Phillip W. Graham, DrPH, Senior Director, Center 
on Behavioral Health, Epidemiology, 
Implementation and Evaluation Research 
Dallas J. Elgin, PhD, Senior Researcher, Child and 
Adolescent Research and Evaluation Program 
RTI International  
RTI International 
3040 East Cornwallis Road 
P.O. Box 12194 
Research Triangle Park, NC 27709-2194 
2950 E. Harmony Road, Suite 390 Fort Collins, 
CO 80528-3431 
919-485-7752 
919-248-8595 
pgraham@rti.org 
delgin@rti.org 
9.1.5 Quality Assurance and Quality Control 
Each clinical site will perform internal quality management of study conduct, data collection, 
documentation and completion. Quality control procedures will be implemented as follows: 
Informed consent. Study staff will review both the documentation of the consenting process as well as a 
percentage of the completed consent documents. This review will evaluate compliance with GCP, 
accuracy, and completeness. Feedback will be provided to the study team to ensure proper consenting 
procedures are followed.  
Source documents and the electronic data. Data will be initially captured on source documents (see 
Section 9.1.7, Data Handling and Record Keeping) and will ultimately be entered into the study database. 
To ensure accuracy, site staff will compare a representative sample of source data against the database, 
targeting key data points in that review. 
Intervention Fidelity. Consistent delivery of the study interventions will be monitored throughout the 
intervention phase of the study. Procedures for ensuring fidelity of intervention delivery are described in 
Section 6.2.1, Interventionist Training and Tracking.  
Protocol Deviations. The study team will review protocol deviations on an ongoing basis and will 
implement corrective actions when the quantity or nature of deviations are deemed to be at a level of 
concern. 
Should independent monitoring become necessary, the PI will provide direct access to all trial-related 
sites, source data/documents, and reports for the purpose of monitoring and auditing by the 
sponsor/funding agency and inspection by local and regulatory authorities. 
9.1.6 Data Handling and Record Keeping  
Data Collection and Management Responsibilities  
Data collection will be the responsibility of the staff at the clinical sites under the supervision of the site 
investigators. The investigators will be responsible for ensuring the accuracy, completeness, legibility, and 
timeliness of the data reported. 
All source documents will be recorded in the electronic case report form. Data recorded in the case report 
form derived from source documents will be consistent with the data recorded on the source documents. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
36 
Clinical data (including adverse events, concomitant medications, and expected adverse reactions data) 
and clinical laboratory data will be stored in an RTI project share within RTI’s HIPAA-compliant network, a 
21 CFR Part 11–compliant data capture system. The data system includes password protection and 
internal quality checks to identify data that appear inconsistent, incomplete, or inaccurate. Clinical data 
will be entered directly from the source documents. 
Study Records Retention  
Study documents will be retained for a minimum of 2 years have elapsed since the formal discontinuation 
of the study intervention. These documents should be retained for a longer period, however, if required 
by local or CDC, NCIPC regulations. No records will be destroyed without the written consent of the 
sponsor/funding agency, if applicable. It is the responsibility of the sponsor/funding agency to inform the 
investigator when these documents no longer need to be retained. 
9.1.7 Protocol Deviations   
This protocol defines a protocol deviation as any non-compliance with the clinical trial protocol, 
International Council on Harmonisation Good Clinical Practice (ICH GCP), or Manual of Procedures 
requirements. The non-compliance may be either on the part of the participant, the investigator, or the 
study site staff. As a result of deviations, corrective actions will be developed by the site and implemented 
promptly.  
These practices are consistent with ICH GCP:  
• Section 4.5 Compliance with Protocol, subsections 4.5.1, 4.5.2, and 4.5.3  
• Section 5.1 Quality Assurance and Quality Control, subsection 5.1.1  
• Section 5.20 Noncompliance, subsections 5.20.1, and 5.20.2.  
It will be the responsibility of the site investigator to use continuous vigilance to identify and report 
deviations within 1 working days of identification of the protocol deviation. All deviations will be 
addressed in study source documents, reported to CDC, NCIPC Program Official and RTI. Protocol 
deviations will be sent to the reviewing IRB1 per their policies. The site investigator will be responsible for 
knowing and adhering to the reviewing IRB requirements. Further details about the handling of protocol 
deviations will be included in the Manual of Procedures. 
9.1.8 Publication and Data Sharing Policy  
RTI, NJPN, RWIJBH IFPR, CDC, and NCIPC intend to disseminate findings regarding the effectiveness of 
SPF+FA, the risk and protective factors through which it impacts substance use and overdoses, and 
conditions that facilitate optimal adoption and implementation. De-identified data will be shared with 
other researchers and stakeholders in accordance with the NCIPC Data Management Plan. The associated 
publication and dissemination plan will involve (1) providing annual progress reports to CDC; (2) publishing 
manuscripts in peer-reviewed journals focused on substance use and ACEs (e.g., Primary Prevention, 
Prevention Science) and implementation science (e.g., Implementation Science Journal); (3) disseminating 
findings in writing and through presentations to and communication with key stakeholders; and (4) 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
37 
presenting study findings at research conferences (such as those hosted by the Society for Prevention 
Research, American Public Health Association, and Society for Implementation Research Collaboration). 
To the extent that SFP+FA is found to be effective, a key element of our dissemination plan will be to 
provide implementation-related resources that can support dissemination of the strategy. RTI, NJPN, and 
RWJBH IFPR will work together to create resources designed to facilitate support for communities wanting 
to implement the strategies. For example, using findings from the current study, we will identify 
implementation contexts and community characteristics associated with high-quality implementation and 
program effectiveness. Lessons learned may also inform future implementation or adaptation of the 
strategy. This information can be converted into an infographic for coalitions implementing SFP, or it could 
be incorporated into an updated technical package or informational brief distributed by CDC.  
Using information from the study aims and the community engagement interviews, we will develop an 
implementation manual, which will provide customized guidance for integrating an FA within SFP. The 
implementation manual will describe program implementation, including stakeholder engagement, 
protocols used, and barriers and facilitators to implementation. The manual will be disseminated through 
RTI Press—an authoritative, respected outlet for bringing peer-reviewed research, analytic tools, and 
technical expertise to national and international audiences. RTI Press is an open-access publisher; 
publications are available free of charge in PDF format and downloadable from www.rti.org/rti-press. The 
final tool, learnings, and recommendations will be widely shared and disseminated through several 
national networks, such as the National Prevention Network and national conference presentations, such 
as the American Public Health Association, the Society for Research Prevention, the American Evaluation 
Association. 
9.1.9 Conflict of Interest Policy 
The independence of this study from any actual or perceived influence is critical. Therefore, any actual 
conflict of interest of persons who have a role in the design, conduct, analysis, publication, or any aspect 
of this trial will be disclosed and managed. Furthermore, persons who have a perceived conflict of interest 
will be required to have such conflicts managed in a way that is appropriate to their participation in the 
design and conduct of this study. The study leadership in conjunction with the CDC, NCIPC has established 
policies and procedures for all study group members to disclose all conflicts of interest and will establish 
a mechanism for the management of all reported dualities of interest. 
9.2 
ADDITIONAL CONSIDERATIONS 
N/A. 
9.3 
PROTOCOL AMENDMENT HISTORY 
Exhibit 10 is intended to capture changes of IRB-approved versions of the protocol, including a description 
of the change and rationale. A Summary of Changes table for the current amendment is located in the 
Protocol Title Page.  

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
38 
Exhibit 10. Protocol Amendment History 
Version 
Date 
Description of Change  
Brief Rationale 
1.0 
 
Not applicable; this is the first version of the protocol. 
N/A 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10 
REFERENCES  
1. 
Kirby, S., Edwards, K., Yu, S., van Gool, K., Powell-Davies, G., Harris-Roxas, B., Gresham, E., 
Harris, M., & Hall, J. (2021, Apr). Improving outcomes for marginalised rural families through a 
care navigator program. Health Promotion Journal of Australia, 32(2), 285-294. 
https://doi.org/10.1002/hpja.348. PMID: 32323411. 
2. 
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012, Mar). Effectiveness-
implementation hybrid designs: combining elements of clinical effectiveness and 
implementation research to enhance public health impact. Medical Care, 50(3), 217-226. 
https://doi.org/10.1097/MLR.0b013e3182408812. PMCID: PMC3731143. PMID: 22310560. 
3. 
Glennerster, R., & Takavarasha, K. (2014). Running randomized evaluations: A practical guide. 
Princeton University Press.  
4. 
Ellis, W. R., & Dietz, W. H. (2017, Sep-Oct). A new framework for addressing adverse childhood 
and community experiences: the Building Community Resilience model. Academic Pediatrics, 
17(7S), S86-S93. https://doi.org/10.1016/j.acap.2016.12.011. PMID: 28865665. 
5. 
New Jersey Department of Children and Families. (2021). NJ ACES: Statewide action plan. 
https://www.chcs.org/media/508_Final-NJ-ACEs-Action-Plan-February-2021.pdf 
6. 
National Conference of State Legislatures. (2021). Adverse childhood experiences.  
7. 
State of New Jersey Department of Human Services. (2021). New Jersey drug and alcohol abuse 
treatment: Substance abuse overview 2020 statewide. 
https://www.state.nj.us/humanservices/dmhas/publications/statistical/Substance%20Abuse%2
0Overview/2020/Statewide.pdf 
8. 
Panchal, N., Garfield, R., Cox, C., & Artiga, S. (2021). Substance use issues are worsening 
alongside access to care. https://www.kff.org/policy-watch/substance-use-issues-are-
worsening-alongside-access-to-care/ 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
39 
9. 
New Jersey. (2021). New Jersey’s response to the opioid overdose epidemic: A year in review 
2020. 
https://d31hzlhk6di2h5.cloudfront.net/20210527/22/2f/f6/c6/860e131630d897a70b5235e0/Fin
al_Opiod_Response_Report_2020.pdf 
10. 
KFF. (2018). Opioid overdose deaths by race/ethnicity. https://www.kff.org/other/state-
indicator/opioid-overdose-deaths-by-
raceethnicity/?dataView=2&currentTimeframe=1&sortModel=%7B%22colId%22:%22Location%2
2,%22sort%22:%22asc%22%7D 
11. 
Panchal, N., Kamal, R., Cox, C., & Garfield, R. (2021). The implications of COVID-19 for mental 
health and substance use. https://www.kff.org/coronavirus-covid-19/issue-brief/the-
implications-of-covid-19-for-mental-health-and-substance-use/ 
12. 
Centers for Disease Control and Prevention. (2019). Preventing adverse childhood experiences 
(ACEs): Leveraging the best available evidence. 
https://www.cdc.gov/violenceprevention/pdf/preventingACES.pdf 
13. 
Spoth, R. L., Redmond, C., & Shin, C. (2000, Dec). Reducing adolescents' aggressive and hostile 
behaviors: randomized trial effects of a brief family intervention 4 years past baseline. Archives 
of Pediatrics and Adolescent Medicine, 154(12), 1248-1257. 
https://doi.org/10.1001/archpedi.154.12.1248. PMID: 11115311. 
14. 
Spoth, R., Redmond, C., & Lepper, H. (1999, Mar). Alcohol initiation outcomes of universal 
family-focused preventive interventions: one- and two-year follow-ups of a controlled study. 
Journal of Studies on Alcohol. Supplement, 13, 103-111. 
https://doi.org/10.15288/jsas.1999.s13.103. PMID: 10225494. 
15. 
Spoth, R., Trudeau, L., Shin, C., Ralston, E., Redmond, C., Greenberg, M., & Feinberg, M. (2013, 
Apr). Longitudinal effects of universal preventive intervention on prescription drug misuse: 
three randomized controlled trials with late adolescents and young adults. American Journal of 
Public Health, 103(4), 665-672. https://doi.org/10.2105/AJPH.2012.301209. PMCID: 
PMC3673263. PMID: 23409883. 
16. 
Spoth, R. L., Redmond, C., & Shin, C. (2001, Aug). Randomized trial of brief family interventions 
for general populations: adolescent substance use outcomes 4 years following baseline. Journal 
of Consulting and Clinical Psychology, 69(4), 627-642. https://doi.org/10.1037//0022-
006x.69.4.627. PMID: 11550729. 
17. 
Chartier, K. G., Negroni, L. K., & Hesselbrock, M. N. (2010, Jan 1). Strengthening family practices 
for Latino families. J Ethn Cult Divers Soc Work, 19(1), 1-17. 
https://doi.org/10.1080/15313200903531982. PMCID: PMC2943633. PMID: 20871785. 
18. 
Coatsworth, J. D., Duncan, L. G., Greenberg, M. T., & Nix, R. L. (2010, Apr 1). Changing parent's 
mindfulness, child management skills and relationship quality with their youth: Results from a 
randomized pilot intervention trial. J Child Fam Stud, 19(2), 203-217. 
https://doi.org/10.1007/s10826-009-9304-8. PMCID: PMC3765025. PMID: 24013587. 
19. 
Gopalan, G., Hooley, C., Winters, A., & Stephens, T. (2019, Jun). Perceptions among child welfare 
staff when modifying a child mental health intervention to be implemented in Child Welfare 
Services. American Journal of Community Psychology, 63(3-4), 366-377. 
https://doi.org/10.1002/ajcp.12309. PMCID: PMC6398163. PMID: 30693961. 
20. 
Kumpfer, K. L., & Brown, J. L. (2019, Nov 18). A parenting behavior intervention (the 
Strengthening Families program) for families: Noninferiority trial of different program delivery 
methods. JMIR Pediatr Parent, 2(2), e14751. https://doi.org/10.2196/14751. PMCID: 
PMC6887825. PMID: 31738176. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
40 
21. 
Kyritsi, K. P., & Bacopoulou, F. (2021). Cultural accommodation of the Strengthening Families 
program for parents and young adolescents 10-14: Greek Phase I and II study. Advances in 
Experimental Medicine and Biology, 1339, 283-299. https://doi.org/10.1007/978-3-030-78787-
5_34. PMID: 35023116. 
22. 
Miller, A. L., Weston, L. E., Perryman, J., Horwitz, T., Franzen, S., & Cochran, S. (2014, Sep). 
Parenting while incarcerated: Tailoring the Strengthening Families program for use with jailed 
mothers. Child Youth Serv Rev, 44, 163-170. https://doi.org/10.1016/j.childyouth.2014.06.013. 
PMCID: PMC4657561. PMID: 26612963. 
23. 
Roehrig, C., & Pradier, C. (2017, Dec 5). Strengthening Families program: key points for the 
French adaptation. Santé Publique, 29(5), 643-653. https://doi.org/10.3917/spub.175.0643 (Cles 
de l?adaptation francaise d?un programme americain de soutien a la parentalite.). PMID: 
29384298. 
24. 
Whitesell, N. R., Mousseau, A. C., Keane, E. M., Asdigian, N. L., Tuitt, N., Morse, B., Zacher, T., 
Dick, R., Mitchell, C. M., & Kaufman, C. E. (2019, Oct). Integrating community-engagement and a 
multiphase optimization strategy framework: Adapting substance use prevention for American 
Indian families. Prevention Science, 20(7), 1136-1146. https://doi.org/10.1007/s11121-019-
01036-y. PMCID: PMC6774198. PMID: 31376058. 
25. 
Pinheiro-Carozzo, N. P., Murta, S. G., Vinha, L., da Silva, I. M., & Fontaine, A. (2021, Jun 15). 
Beyond effectiveness of the Strengthening Families Program (10-14): a scoping RE-AIM-based 
review. Psicol Reflex Crit, 34(1), 16. https://doi.org/10.1186/s41155-021-00182-z. PMCID: 
PMC8206301. PMID: 34131838. 
26. 
Pfadenhauer, L. M., Gerhardus, A., Mozygemba, K., Lysdahl, K. B., Booth, A., Hofmann, B., 
Wahlster, P., Polus, S., Burns, J., Brereton, L., & Rehfuess, E. (2017, Feb 15). Making sense of 
complexity in context and implementation: the Context and Implementation of Complex 
Interventions (CICI) framework. Implement Sci, 12(1), 21. https://doi.org/10.1186/s13012-017-
0552-5. PMCID: PMC5312531. PMID: 28202031. 
27. 
Desveaux, L., McBrien, K., Barnieh, L., & Ivers, N. M. (2019, Jan 8). Mapping variation in 
intervention design: a systematic review to develop a program theory for patient navigator 
programs. Syst Rev, 8(1), 8. https://doi.org/10.1186/s13643-018-0920-5. PMCID: PMC6323765. 
PMID: 30621796. 
28. 
McBrien, K. A., Ivers, N., Barnieh, L., Bailey, J. J., Lorenzetti, D. L., Nicholas, D., Tonelli, M., 
Hemmelgarn, B., Lewanczuk, R., Edwards, A., Braun, T., & Manns, B. (2018). Patient navigators 
for people with chronic disease: A systematic review. PloS One, 13(2), e0191980. 
https://doi.org/10.1371/journal.pone.0191980. PMCID: PMC5819768. PMID: 29462179. 
29. 
Stitzer, M., Matheson, T., Cunningham, C., Sorensen, J. L., Feaster, D. J., Gooden, L., Hammond, 
A. S., Fitzsimons, H., & Metsch, L. R. (2017, Jun 27). Enhancing patient navigation to improve 
intervention session attendance and viral load suppression of persons with HIV and substance 
use: a secondary post hoc analysis of the Project HOPE study. Addiction Science & Clinical 
Practice, 12(1), 16. https://doi.org/10.1186/s13722-017-0081-1. PMCID: PMC5485550. PMID: 
28651612. 
30. 
Summers, N. A., Colasanti, J. A., Feaster, D. J., Armstrong, W. S., Rodriguez, A., Jain, M. K., 
Jacobs, P., Metsch, L. R., & Del Rio, C. (2020, May). Predictors for poor linkage to care among 
hospitalized persons living with HIV and co-occurring substance use disorder. AIDS Research and 
Human Retroviruses, 36(5), 406-414. https://doi.org/10.1089/AID.2019.0153. PMCID: 
PMC7232662. PMID: 31914790. 
31. 
Radigan, M., Wang, R., Chen, Y., & Xiang, J. (2014, Nov). Youth and caregiver access to peer 
advocates and satisfaction with mental health services. Community Mental Health Journal, 
50(8), 915-921. https://doi.org/10.1007/s10597-014-9709-8. PMID: 24522553. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
41 
32. 
SAMHSA’s Center for the Application of Prevention Technologies. (2018). The role of adverse 
childhood experiences in substance misuse and related behavioral health problems. 
https://mnprc.org/wp-content/uploads/2019/01/aces-behavioral-health-problems.pdf 
33. 
Eriksson, M., Ghazinour, M., & Hammarström, A. (2018). Different uses of Bronfenbrenner’s 
ecological theory in public mental health research: what is their value for guiding public mental 
health policy and practice? Social Theory & Health, 16(4), 414-433. 
https://doi.org/10.1057/s41285-018-0065-6. 
34. 
Bryan, R. H. (2019). Getting to Why: Adverse childhood experiences' impact on adult health. The 
Journal for Nurse Practitioners, 15(2), 153-157.e151. 
https://doi.org/10.1016/j.nurpra.2018.09.012. 
35. 
Mersky, J. P., Topitzes, J., & Reynolds, A. J. (2013, Nov). Impacts of adverse childhood 
experiences on health, mental health, and substance use in early adulthood: a cohort study of 
an urban, minority sample in the U.S. Child Abuse and Neglect, 37(11), 917-925. 
https://doi.org/10.1016/j.chiabu.2013.07.011. PMCID: PMC4090696. PMID: 23978575. 
36. 
Vein, M. (2019). 3 realms of ACEs handout. PACEs connection resource center. 
https://www.pacesconnection.com/g/resource-center/blog/3-realms-of-aces-handout 
37. 
Brown, D. W., Anda, R. F., Tiemeier, H., Felitti, V. J., Edwards, V. J., Croft, J. B., & Giles, W. H. 
(2009, Nov). Adverse childhood experiences and the risk of premature mortality. American 
Journal of Preventive Medicine, 37(5), 389-396. https://doi.org/10.1016/j.amepre.2009.06.021. 
PMID: 19840693. 
38. 
Douglas, K. R., Chan, G., Gelernter, J., Arias, A. J., Anton, R. F., Weiss, R. D., Brady, K., Poling, J., 
Farrer, L., & Kranzler, H. R. (2010, Jan). Adverse childhood events as risk factors for substance 
dependence: partial mediation by mood and anxiety disorders. Addictive Behaviors, 35(1), 7-13. 
https://doi.org/10.1016/j.addbeh.2009.07.004. PMCID: PMC2763992. PMID: 19720467. 
39. 
Giano, Z., Wheeler, D. L., & Hubach, R. D. (2020, Sep 10). The frequencies and disparities of 
adverse childhood experiences in the U.S. BMC Public Health, 20(1), 1327. 
https://doi.org/10.1186/s12889-020-09411-z. PMCID: PMC7488299. PMID: 32907569. 
40. 
Iniguez, K. C., & Stankowski, R. V. (2016, Dec). Adverse childhood experiences and health in 
adulthood in a rural population-based sample. Clinical Medicine & Research, 14(3-4), 126-137. 
https://doi.org/10.3121/cmr.2016.1306. PMCID: PMC5302459. PMID: 27503793. 
41. 
Dube, S. R., Anda, R. F., Felitti, V. J., Chapman, D. P., Williamson, D. F., & Giles, W. H. (2001, Dec 
26). Childhood abuse, household dysfunction, and the risk of attempted suicide throughout the 
life span: findings from the Adverse Childhood Experiences Study. JAMA, 286(24), 3089-3096. 
https://doi.org/10.1001/jama.286.24.3089. PMID: 11754674. 
42. 
Center for Youth Wellness. (2014). A hidden crisis: Findings on adverse childhood experiences in 
California. https://centerforyouthwellness.org/wp-content/themes/cyw/build/img/building-a-
movement/hidden-crisis.pdf 
43. 
Ege, M. A., Messias, E., Thapa, P. B., & Krain, L. P. (2015, Jan). Adverse childhood experiences 
and geriatric depression: results from the 2010 BRFSS. American Journal of Geriatric Psychiatry, 
23(1), 110-114. https://doi.org/10.1016/j.jagp.2014.08.014. PMCID: PMC4267899. PMID: 
25306195. 
44. 
Haynes, E., Crouch, E., Probst, J., Radcliff, E., Bennett, K., & Glover, S. (2020). Exploring the 
association between a parent’s exposure to Adverse Childhood Experiences (ACEs) and 
outcomes of depression and anxiety among their children. Children and Youth Services Review, 
113. https://doi.org/10.1016/j.childyouth.2020.105013. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
42 
45. 
Gilchrist, G., Radcliffe, P., Noto, A. R., & d'Oliveira, A. F. (2017, Jan). The prevalence and factors 
associated with ever perpetrating intimate partner violence by men receiving substance use 
treatment in Brazil and England: A cross-cultural comparison. Drug and Alcohol Review, 36(1), 
34-51. https://doi.org/10.1111/dar.12436. PMID: 27709693. 
46. 
Hughes, K., Lowey, H., Quigg, Z., & Bellis, M. A. (2016, Mar 3). Relationships between adverse 
childhood experiences and adult mental well-being: results from an English national household 
survey. BMC Public Health, 16, 222. https://doi.org/10.1186/s12889-016-2906-3. PMCID: 
PMC4778324. PMID: 26940088. 
47. 
Swedo, E. A., Sumner, S. A., de Fijter, S., Werhan, L., Norris, K., Beauregard, J. L., Montgomery, 
M. P., Rose, E. B., Hillis, S. D., & Massetti, G. M. (2020, Sep). Adolescent opioid misuse 
attributable to adverse childhood experiences. Journal of Pediatrics, 224, 102-109 e103. 
https://doi.org/10.1016/j.jpeds.2020.05.001. PMCID: PMC8253221. PMID: 32437756. 
48. 
Forster, M., Gower, A. L., Borowsky, I. W., & McMorris, B. J. (2017, May). Associations between 
adverse childhood experiences, student-teacher relationships, and non-medical use of 
prescription medications among adolescents. Addictive Behaviors, 68, 30-34. 
https://doi.org/10.1016/j.addbeh.2017.01.004. PMID: 28088740. 
49. 
Carlson, R. G., Nahhas, R. W., Martins, S. S., & Daniulaityte, R. (2016, Mar 1). Predictors of 
transition to heroin use among initially non-opioid dependent illicit pharmaceutical opioid users: 
A natural history study. Drug and Alcohol Dependence, 160, 127-134. 
https://doi.org/10.1016/j.drugalcdep.2015.12.026. PMCID: PMC4767561. PMID: 26785634. 
50. 
Cerda, M., Santaella, J., Marshall, B. D., Kim, J. H., & Martins, S. S. (2015, Sep). Nonmedical 
prescription opioid use in childhood and early adolescence predicts transitions to heroin use in 
young adulthood: a national study. Journal of Pediatrics, 167(3), 605-612 e601-602. 
https://doi.org/10.1016/j.jpeds.2015.04.071. PMCID: PMC4714948. PMID: 26054942. 
51. 
Grella, C. E., & Lovinger, K. (2011, Nov 1). 30-year trajectories of heroin and other drug use 
among men and women sampled from methadone treatment in California. Drug and Alcohol 
Dependence, 118(2-3), 251-258. https://doi.org/10.1016/j.drugalcdep.2011.04.004. PMCID: 
PMC3156933. PMID: 21549528. 
52. 
McCabe, S. E., West, B. T., Morales, M., Cranford, J. A., & Boyd, C. J. (2007, Dec). Does early 
onset of non-medical use of prescription drugs predict subsequent prescription drug abuse and 
dependence? Results from a national study. Addiction, 102(12), 1920-1930. 
https://doi.org/10.1111/j.1360-0443.2007.02015.x. PMCID: PMC2377405. PMID: 17916222. 
53. 
Turney, K., & Olsen, A. (2019, Apr). Household member substance problems and children's 
health in the United States. SSM Popul Health, 7, 100400. 
https://doi.org/10.1016/j.ssmph.2019.100400. PMCID: PMC6517526. PMID: 31193083. 
54. 
Hedegaard, H., Miniño, A. M., Spencer, M. R., & Warner, M. (2021). Drug overdose deaths in the 
United States, 1999–2020. Hyattsville, MD, CDC National Center for Health Statistics. 
https://www.cdc.gov/nchs/data/databriefs/db428.pdf 
55. 
Centers for Disease Control and Prevention. Risk and protective factors. 
https://www.cdc.gov/violenceprevention/aces/riskprotectivefactors.html 
56. 
Coulton, C. J., Korbin, J. E., & Su, M. (1999, Nov). Neighborhoods and child maltreatment: a 
multi-level study. Child Abuse and Neglect, 23(11), 1019-1040. https://doi.org/10.1016/s0145-
2134(99)00076-9. PMID: 10604060. 
57. 
Maguire-Jack, K., & Negash, T. (2016). Parenting stress and child maltreatment: The buffering 
effect of neighborhood social service availability and accessibility. Children and Youth Services 
Review, 60, 27-33. https://doi.org/10.1016/j.childyouth.2015.11.016. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
43 
58. 
Molnar, B. E., Goerge, R. M., Gilsanz, P., Hill, A., Subramanian, S. V., Holton, J. K., Duncan, D. T., 
Beatriz, E. D., & Beardslee, W. R. (2016, Jan). Neighborhood-level social processes and 
substantiated cases of child maltreatment. Child Abuse and Neglect, 51, 41-53. 
https://doi.org/10.1016/j.chiabu.2015.11.007. PMCID: PMC4713333. PMID: 26684963. 
59. 
Cuellar, J., Jones, D. J., & Sterrett, E. (2015, Jan). Examining parenting in the neighborhood 
context: a review. J Child Fam Stud, 24(1), 195-219. https://doi.org/10.1007/s10826-013-9826-y. 
PMCID: PMC4573634. PMID: 26392738. 
60. 
Downey, L., & Van Willigen, M. (2005, Sep). Environmental stressors: the mental health impacts 
of living near industrial activity. Journal of Health and Social Behavior, 46(3), 289-305. 
https://doi.org/10.1177/002214650504600306. PMCID: PMC3162363. PMID: 16259150. 
61. 
Schofield, T. J., Donnellan, M. B., Merrick, M. T., Ports, K. A., Klevens, J., & Leeb, R. (2018, Sep). 
Intergenerational continuity in adverse childhood experiences and rural community 
environments. American Journal of Public Health, 108(9), 1148-1152. 
https://doi.org/10.2105/AJPH.2018.304598. PMCID: PMC6085037. PMID: 30089003. 
62. 
Valentino, K., Nuttall, A. K., Comas, M., Borkowski, J. G., & Akai, C. E. (2012, May). 
Intergenerational continuity of child abuse among adolescent mothers: authoritarian parenting, 
community violence, and race. Child Maltreatment, 17(2), 172-181. 
https://doi.org/10.1177/1077559511434945. PMID: 22287568. 
63. 
Chaudry, A., & Wimer, C. (2016, Apr). Poverty is not just an indicator: the relationship between 
income, poverty, and child well-being. Academic Pediatrics, 16(3 Suppl), S23-29. 
https://doi.org/10.1016/j.acap.2015.12.010. PMID: 27044698. 
64. 
Dreyer, B., Chung, P. J., Szilagyi, P., & Wong, S. (2016, Apr). Child poverty in the united states 
today: Introduction and Executive Summary. Academic Pediatrics, 16(3 Suppl), S1-5. 
https://doi.org/10.1016/j.acap.2016.02.010. PMID: 27044686. 
65. 
The Annie E. Casey Foundation. (2021). 2021 Kids Count data book: 2021 state trends in child 
well-being. 
66. 
Orsi, R., Yuma-Guerrero, P., Sergi, K., Pena, A. A., & Shillington, A. M. (2018, Dec). Drug overdose 
and child maltreatment across the United States' rural-urban continuum. Child Abuse and 
Neglect, 86, 358-367. https://doi.org/10.1016/j.chiabu.2018.08.010. PMID: 30166067. 
67. 
Karriker-Jaffe, K. J. (2011, Jan). Areas of disadvantage: a systematic review of effects of area-
level socioeconomic status on substance use outcomes. Drug and Alcohol Review, 30(1), 84-95. 
https://doi.org/10.1111/j.1465-3362.2010.00191.x. PMCID: PMC3057656. PMID: 21219502. 
68. 
Lundahl, L. H., & Cannoy, C. (2021, Oct). COVID-19 and substance use in adolescents. Pediatric 
Clinics of North America, 68(5), 977-990. https://doi.org/10.1016/j.pcl.2021.05.005. PMCID: 
PMC8445753. PMID: 34538307. 
69. 
Wilkins, N., Tsao, B., Hertz, M. F., Davis, R., & Klevens, J. (2014). Connecting the dots: An 
overview of the links among multiple forms of violence. Atlanta, GA and Oakland, CA, National 
Center for Injury Prevention and Control, Centers for Disease Control and Prevention, 
Prevention Institute. https://stacks.cdc.gov/view/cdc/31552 
70. 
Galea, S., Hall, C., & Kaplan, G. A. (2009, May). Social epidemiology and complex system dynamic 
modelling as applied to health behaviour and drug use research. International Journal on Drug 
Policy, 20(3), 209-216. https://doi.org/10.1016/j.drugpo.2008.08.005. PMCID: PMC2782722. 
PMID: 18930649. 
71. 
Lines, L. M., Long, M. C., Humphrey, J. L., Nguyen, C. T., Scanlon, S., Berzin, O. K. G., Brown, M. 
C., & Bir, A. (2022). Artificially intelligent social risk adjustment: Development and pilot testing in 
Ohio. (RTI Press Research Report No. RR-0047-2209, RTI Press. 
https://doi.org/10.3768/rtipress.2022.rr.0047.2209 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
44 
72. 
Healthcare Cost and Utilization Project (HCUP). (2005-2009). HCUP State Inpatient databases 
(SID). Rockville, MD, Agency for Healthcare Research and Quality. www.hcup-
us.ahrq.gov/sidoverview.jsp 
73. 
Healthcare Cost and Utilization Project (HCUP). (2009). HCUP State Emergency Department 
databases (SEDD). Rockville, MD, Agency for Healthcare Research and Quality. www.hcup-
us.ahrq.gov/seddoverview.jsp 
74. 
Children's Bureau, Administration On Children, Youth And Families, Administration For Children 
And Families, U. S. Department Of Health And Human Services. (2012). National Child Abuse and 
Neglect Data System (NCANDS) Child File (Dataset 178; Version FFY 2012). National Data Archive 
on Child Abuse and Neglect. https://doi.org/Https://doi.org/10.34681/DKTJ-KM27 
75. 
Gulliford, M. C., Ukoumunne, O. C., & Chinn, S. (1999, May 1). Components of variance and 
intraclass correlations for the design of community-based surveys and intervention studies: data 
from the Health Survey for England 1994. American Journal of Epidemiology, 149(9), 876-883. 
https://doi.org/10.1093/oxfordjournals.aje.a009904. PMID: 10221325. 
76. 
Adams, G., Gulliford, M. C., Ukoumunne, O. C., Eldridge, S., Chinn, S., & Campbell, M. J. (2004, 
Aug). Patterns of intra-cluster correlation from primary care research to inform study design and 
analysis. Journal of Clinical Epidemiology, 57(8), 785-794. 
https://doi.org/10.1016/j.jclinepi.2003.12.013. PMID: 15485730. 
77. 
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: applications and data 
analysis methods (2nd ed.). Sage Publications, Inc.  
78. 
Dimick, J. B., & Ryan, A. M. (2014, Dec 10). Methods for evaluating changes in health care policy: 
the difference-in-differences approach. JAMA, 312(22), 2401-2402. 
https://doi.org/10.1001/jama.2014.16153. PMID: 25490331. 
79. 
Duncan, T. E., & Duncan, S. C. (2004). An introduction to latent growth curve modeling. Behavior 
Therapy, 35(2), 333-363. https://doi.org/10.1016/s0005-7894(04)80042-x. 
80. 
Hox, J. (2002 ). Multilevel analysis: Techniques and applications Psychology Press.  
81. 
Kwok, O. M., Underhill, A. T., Berry, J. W., Luo, W., Elliott, T. R., & Yoon, M. (2008, Aug). 
Analyzing longitudinal data with multilevel models: An example with individuals living with 
lower extremity intra-articular fractures. Rehabilitation Psychology, 53(3), 370-386. 
https://doi.org/10.1037/a0012765. PMCID: PMC2613314. PMID: 19649151. 
82. 
Wing, C., Simon, K., & Bello-Gomez, R. A. (2018, Apr 1). Designing difference in difference 
studies: Best practices for public health policy research. Annual Review of Public Health, 39, 453-
469. https://doi.org/10.1146/annurev-publhealth-040617-013507. PMID: 29328877. 
83. 
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and 
event occurrence. Oxford university press.  
84. 
Enders, C. K. (2001). A primer on maximum likelihood algorithms available for use with missing 
data. Structural Equation Modeling: A Multidisciplinary Journal, 8(1), 128-141. 
https://doi.org/10.1207/s15328007sem0801_7. 
85. 
Graham, J. W. (2003). Adding missing-data-relevant variables to FIML-based structural equation 
models. Structural Equation Modeling: A Multidisciplinary Journal, 10(1), 80-100. 
https://doi.org/10.1207/s15328007sem1001_4. 
86. 
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). John Wiley.  
87. 
Little, R. J. A., & Schenker, N. (1995). Missing data. In G. Arminger, C. C. Clogg, & M. E. Sobel 
(Eds.), Handbook of statistical modeling for the social and behavioral sciences (pp. 39–75). 
Plenum Press.  
88. 
Schafer, J. L. (1997). Analysis of incomplete multivariate data. Chapman & Hall.  
89. 
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. 
Psychological Methods, 7(2), 147-177. https://doi.org/10.1037/1082-989x.7.2.147. 

Evaluating an ACE-Targeting Advocate Model of a Substance Use Prevention Program 
Version 1.0  
Protocol # 0218779 
August 31, 2023 
45 
90. 
Verbeke, G., & Molenberghs, G. (2013). Linear mixed models for longitudinal data. Springer.  
91. 
Kennedy, D. P., Osilla, K. C., Hunter, S. B., Golinelli, D., Maksabedian Hernandez, E., & Tucker, J. 
S. (2022). Restructuring personal networks with a Motivational Interviewing social network 
intervention to assist the transition out of homelessness: A randomized control pilot study. PLoS 
ONE, 17(1): e0262210. 
92. 
Wölfer, R., Faber, N. S., & Hewstone, M. (2015). Social network analysis in the science of groups: 
Cross-sectional and longitudinal applications for studying intra-and intergroup behavior. Group 
Dynamics: Theory, Research, and Practice, 19(1), 45. 
93. 
Johnson, E. M., & Chew, R. (2021). Social network analysis methods for international 
development. RTI Press. 
94. 
Noldus, R., & Van Mieghem, P. (2015). Assortativity in complex networks. Journal of Complex 
Networks, 3(4), 507-542. 
95. 
Opsahl, T. (2013). Triadic closure in two-mode networks: Redefining the global and local 
clustering coefficients. Social networks, 35(2), 159-167. 
96. 
Jordan, D. G., Winer, E. S., & Salem, T. (2020). The current status of temporal network analysis 
for clinical science: Considerations as the paradigm shifts? Journal of clinical psychology, 76(9), 
1591-1612. 
97. 
Lusher, D., Koskinen, J., & Robins, G. (Eds.). (2013). Exponential random graph models for social 
networks: Theory, methods, and applications. Cambridge University Press. 
98. 
Spoth, R. L., Guyll, M., & Day, S. X. (2002, Mar). Universal family-focused interventions in 
alcohol-use disorder prevention: cost-effectiveness and cost-benefit analyses of two 
interventions. Journal of Studies on Alcohol, 63(2), 219-228. 
https://doi.org/10.15288/jsa.2002.63.219. PMID: 12033699. 
99. 
Suter, C. R. (2010). Economic evaluation of a community-based, family-skills prevention program 
[Masters thesis, Washington State University].  
100. 
Creswell, J. W., & Clark, V. L. P. (2017). Choosing a mixed methods design. In Designing and 
conducting mixed methods research. Sage publications.  
101. 
Hanson, W. E., Creswell, J. W., Clark, V. L. P., Petska, K. S., & Creswell, J. D. (2005). Mixed 
methods research designs in counseling psychology. Journal of Counseling Psychology, 52(2), 
224-235. https://doi.org/10.1037/0022-0167.52.2.224. 
102. 
Jick, T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. 
Administrative Science Quarterly, 24(4). https://doi.org/10.2307/2392366. 
103. 
Riccucci, N. M. (2010). Public administration: Traditions of inquiry and philosophies of 
knowledge. Georgetown University Press.  
104. 
Guetterman, T. C., Fetters, M. D., & Creswell, J. W. (2015, Nov). Integrating quantitative and 
qualitative results in health science mixed methods research through joint displays. Annals of 
Family Medicine, 13(6), 554-561. https://doi.org/10.1370/afm.1865. PMCID: PMC4639381. 
PMID: 26553895.