Comparative Effectiveness Trail to Reduce Child Maltreatment, Improve Client Outcomes and Examine Client Burden
Phase: NA
Conditions: Child Maltreatment
Sponsor: Daniel Whitaker

1 
 
 
 
 
Research Protocol  
 
 
 
Comparative Effectiveness Trial to Reduce Child Maltreatment, 
Improve Client Outcomes and Examine Client Burden 
 
 
NCT02549287 
11/17/16 
 
 
 
 
CER-1409-21178, Patient Centered Outcomes Research Institute 
 
 
Georgia State University,  
Daniel J. Whitaker, PhD, PI 
 

2 
 
Contents 
A. 
PROJECT SUMMARY ............................................................................................................... 3 
B. 
BACKGROUND ....................................................................................................................... 4 
B1. General Project information .................................................................................................... 4 
B2. Study goals and objectives ....................................................................................................... 5 
B3. Background and rationale for study ......................................................................................... 5 
C. 
METHOD ................................................................................................................................ 7 
C1. Site Recruitment ...................................................................................................................... 7 
C2. Site Application and selection ................................................................................................. 8 
C3. Interventions ............................................................................................................................ 8 
C3.1 
Supportive Case Management .................................................................................................. 8 
C3.2 
SafeCare ................................................................................................................................... 9 
C3.3 
Tracking interventions received and utilization ..................................................................... 10 
C4. Aim 1. Identify and assess barriers and facilitators that influence implementation of a 
structured behavioral parenting program ......................................................................................... 10 
C4.1 
Overview ................................................................................................................................ 10 
C4.2 
Qualitative .............................................................................................................................. 11 
Provider, supervisor, and administrative recruitment ......................................................................... 11 
C5. Aim 2. Understand parent and provider reactions to SC and SCM services, especially parent 
perceptions related to trajectory of burden, satisfaction, and perceived impact across intervention 
receipt 11 
C5.1 
Family burden, satisfaction and perceived impact ................................................................. 11 
C5.2 
Provider burden, satisfaction and perceived impact ............................................................... 13 
C6. Aim 3. Compare SC to SCM on parent outcomes and child outcomes including parenting 
skill, parenting stress, and parent-child relationship, and child behavioral, social and emotional 
well-being ........................................................................................................................................ 14 
C6.1 
Target population ................................................................................................................... 14 
C6.2 
Family Recruitment ................................................................................................................ 14 
C6.3 
Family assessments of outcomes............................................................................................ 14 
C7. Data Management .................................................................................................................. 16 
C8. Data cleaning and missing data ............................................................................................. 16 
C9. Statistical power for quantitative analyses ............................................................................. 16 
C10. Ethical Considerations ........................................................................................................... 18 
D. 
REFERENCES........................................................................................................................ 19 
 
 
 
 
 

3 
 
 
A. PROJECT SUMMARY  
Child maltreatment is a significant public health problem worldwide. National data indicate that 
in 2012, there were over 3.4 million reports to child protective service systems for suspected 
child maltreatment on over 6.3 million children. Official reports represent only cases reported to 
local CPS systems, however, and the number of cases of maltreatment is likely much higher. 
Child maltreatment leads to a number of negative health impacts. Social and psychological 
impacts of maltreatment are common and pervasive. Maltreated children have more problems 
than non-maltreated children in many areas, including anxiety, depression, aggression, poor 
school performance (Fergusson, Boden, & Horwood, 2008; Gilbert et al., 2009; Herrenkohl, 
Herrenkohl, Rupert, Egolf, & Lutz, 1995; Thornberry, Ireland, & Smith, 2001)  and these 
problems often persist into adulthood.  Findings among adults suggest that maltreatment is 
related to health conditions such as autoimmune diseases, lung cancer, heart disease, and a lower 
health-related quality of life (Felitti et al., 1998). Across health outcomes, maltreatment raises 
risk of experiencing negative outcomes by about 70% (Wegman & Stetler, 2009).  
Most intervention studies conducted within child welfare have focused on safety and 
permanence: whether children experience repeated maltreatment and whether they are removed 
from the home. Little focus has been placed on families’ experiences of services, or outcomes 
they might identify as important.   
The goal of this project is to conduct a comparative effectiveness trial of two different 
interventions for parents referred by child protective service systems. The first intervention is 
supportive case management (SCM), which is a process oriented service aimed at building 
rapport, providing family support, crisis management, and provision of referrals to additional 
services.  The second intervention is behavioral parent training using the SafeCare protocol, 
which focuses on teaching specific skills that help parents to avoid maltreatment. SCM is the 
dominant intervention model in child welfare settings, but behavioral parenting has shown 
promise in reducing recidivism of maltreatment. Neither has been extensively evaluated with 
respect to client outcomes of well-being, or service delivery burden, satisfaction, and perceived 
effectiveness.   
The significance of this study is threefold.  First, with multiple sites, the study will represent one 
of the broadest studies ever conducted in a child welfare setting.  Second, the study will have a 
strong focus on metrics of family well-being, which are typically given little attention in child 
welfare research. This focus has the potential to uncover impacts that were not previously 
measured, and to change the way child welfare systems think about assessing intervention 
impact.  Last, we will focus strongly on parents’ and providers’ experiences of these 
interventions.  This is key because child welfare services are often involuntary, and parent 
engagement in services can be difficult.  By assessing how parents and providers experience 
different interventions, we can better implement those interventions in a way that engages more 
parents.  
 
 

4 
 
B. BACKGROUND 
B1. General Project information  
Title:  Comparative Effectiveness Trial to Reduce Child Maltreatment, Improve Client 
Outcomes and Examine Client Burden 
Sponsor:  Patient Centered Outcomes Research Institute, CER# 1409-21178  
Research Site: Georgia State University, School of Public Health, Center for Healthy 
Development, National SafeCare Training and Research Center, P.O. Box 3995, Atlanta, Ga, 
30302-3995 
Principal Investigator:  Daniel Whitaker, Ph.D., Professor, Director, Division of Health 
Promotion and Behavior, School of Public Health, Georgia State University, Director, National 
SafeCare Training and Research Center, dwhitaker@gsu.edu, 404-413-1282 
Co-investigators:  
Matthew Hayat, Ph.D., Associate Professor, Division of Epidemiology and Biostatistics, School 
of Public Health, Georgia State University, mhayat@gsu.edu, 404-413-9314  
John Lutzker, Ph.D., Distinguished University Professor, Associate Dean, School of Public 
Health, Director, Center for Healthy Development, jlutzker@gsu.edu, 404-413-1284 
Shannon Self-Brown, Ph.D., Professor, School of Public Health, Director, School of Public 
Health Ph.D. Program, Associate Director, National SafeCare Training and Research Center, 
sselfbrown@gsu.edu, 404-413-1283 
Jenelle Shanley-Chatham, Ph.D., Assistant Professor, School of Public Health, Georgia State 
University, Associate Director of Training, National SafeCare Training and Research Center, 
jshanley@gsu.edu, 404-413-1340    
Cathleen Willging, Ph.D., Senior Scientist, Pacific Institute for Research and Evaluation, 
Albuquerque, New Mexico, CWillging@PIRE.org   
Research Staff: 
Colleen McCarty, MPH, Research Coordinator, National SafeCare Training and Research 
Center, cmccarty3@gsu.edu, 404-413-1294  
Erin McFry, MPH, Research Coordinator, National SafeCare Training and Research Center, 
emcfry1@gsu.edu, 404-413-1298 
Matthew Lyons, Graduate Research Assistant, National SafeCare Training and Research Center, 
mlyons5@student.gsu.edu  
Katie Franchot, Graduate Research Assistant, National SafeCare Training and Research Center, 
kfranchot1@student.gsu.edu   
Patient Partners  
Jessica Brown, President, Foster Love, Atlanta, Georgia, jsrbrown@gmail.com  

5 
 
Susanna Patterson, Executive Director, Middle Georgia CASA, Macon, Georgia, 
susanna.patterson@cgcasa.org  
Jamy Still, Groundskeeper, Georgia State Capital, Atlanta, Georgia, kingjaim84@gmail.com  
Stakeholder Partners   
Nicole Perez, Service Provider, Parent Aid, Inc., Tucson, Arizona, nicole@parentaid.org  
Julia Neighbors, JD, Director, Prevent Child Abuse Georgia, Atlanta, Georgia, 
jneighbors@gsu.edu 
Kimberlee Shoecraft, Mentor, The Incredible Years Inc., Seattle, Washington, 
kimber75@hotmail.com    
B2. Study goals and objectives  
The study is designed as a multisite comparative effectiveness trial with randomization of teams 
of providers at each of 8 sites to SCM or SafeCare.  We will recruit sites already conducting 
SCM, and randomize teams to continue SCM at the site, or to implement SafeCare. The study 
aims are as follows: 
(1) Primary Aim 1: To identify and assess barriers and facilitators of implementation of a 
structured behavioral parenting program SafeCare (SC). 
(2) Primary Aim 2:  Understand parent and provider reactions to SC services and Supportive 
Case Management (SCM), especially parents’ perceptions related to trajectory of burden, 
engagement, satisfaction, and perceived impact across intervention receipt.  
(3) Secondary Aim 3:  To examine the short-term impact of SC versus SCM on client-centered 
outcomes, namely parenting variables, parent mental health and well-being, and child 
behavioral, social, and emotional well-being.   
 
B3.  Background and rationale for study 
Primary Aim 1.  Identify and assess barriers and facilitators that influence implementation of 
a structured behavioral parenting program.  
In child welfare services, structured behavioral parenting programs have been documented to 
reduce important child-welfare outcomes, including child maltreatment recidivism (Chaffin, 
Funderburk, Bard, Valle, & Gurwitch, 2011; Chaffin, Hecht, Bard, Silovsky, & Beasley, 2012; 
Prinz, Sanders, Shapiro, Whitaker, & Lutzker, 2009). Thus, from a child welfare system 
perspective, there is a clear rationale for the broad dissemination and effective implementation of 
such programs within child welfare systems.  Many child welfare systems have adopted or 
attempted to adopt one or more evidence-based behavioral parenting programs with varying 
levels of success.  
Theoretical models of implementation such as Exploration, Preparation, Implementation, and 
Sustainment conceptual model (Aarons, Hurlburt, & Horwitz, 2011) suggest systems, 
organizational, and individual variables that can positively or negatively affect implementation 
of evidence-based interventions in complex human service systems.  In this model, many of the 

6 
 
drivers and barriers to implementation may be attributable to system-specific variables, and/or 
site-specific because of organizational and individual (e.g., provider) variables. The current 
project affords us a unique opportunity to study how this range of factors can impact 
implementation of SC in a diversity of child-welfare serving sites and systems.   
Given our sample of ten sites (including one site that failed to implemented SC), we have a 
unique opportunity to conduct research that will inform implementation challenges across 
several different sites, systems, and states. The majority of all other implementation studies 
largely focus on how SC is carried out in a single state or child welfare system. Our study will 
contribute significantly to the available literature by illuminating key commonalities or 
differences across diverse settings that impinge on effective implementation of an evidence-
based intervention. Our research design and data collection will be organized according to the 
EPIS conceptual model outlined by Aarons and colleagues (Aarons et al., 2011). More 
specifically, the EPIS conceptual model points to several classes of factors that can influence 
implementation across developmental phases of implementation, which we will investigate in 
greater depth. These include outer-context factors, such as funding and inter-organizational 
networks, and inner-context factors such as organizational culture, climate, and leadership. We 
will apply the EPIS conceptual model to examine these factors and other potential sources of 
influence on implementation.  
We will use a qualitative research strategy that includes semi-structured interviews and focus 
groups with several levels of staff responsible for implementing the model: program 
administrators, supervisors, and frontline staff (providers). This strategy will yield insight into an 
array of stakeholder perspectives concerning implementation.  We will also conduct interviews 
with the training staff at National SafeCare Training and Research Center (NSTRC) who oversee 
the implementation at each site. In contrast to most implementation studies, ours will be the first 
to conduct a complementary set of interviews and focus groups with staff who have not been 
trained in SC, the SCM providers. This second set of interviews and focus groups will serve two 
primary purposes:  (1) to understand typical barriers to service delivery (not model specific) that 
are commonly experienced in order to identify unique barriers and facilitators to SC 
implementation; and (2) to assess from provider perspectives key factors that must be prioritized 
and addressed when implementing a new evidence-based program.  These two foci will help us 
differentiate barriers and facilitators that may be associated largely with implementation of a new 
model from those that generally exist at a service agency or at the larger system level, and help 
to prepare for future expansions of SC at both the study sites as well as other settings.  
Primary Aim 2.  Understand parent and provider reactions to SC and SCM services, especially 
parent perceptions related to trajectory of burden, satisfaction, and perceived impact across 
intervention receipt 
In testing intervention for child-welfare referred clients, most research has focused on the 
important outcomes mandated by child-welfare legislation, safety, permanence, and well-being 
(but, especially safety and permanence).  There has been little effort to understand client 
responses or preferences for different.  This is surprising because (1) clients can be and often are 
mandated to services and thus demonstrate resistance to services, and (2) program attrition in 
child welfare referred serviced has been documented to be very high, and poor service 
engagement and completion are critical barriers in child welfare service and research. 

7 
 
Because of these issues, it is critical to understand client perceptions of intervention services. In 
particular, it is key to understand client goals, satisfaction, and perceived benefit/effectiveness to 
understand client engagement.  As noted in the original application, most child welfare services, 
including SCM are fairly unstructured, consisting of emotional support, service referrals, crisis 
management, and didactic parenting advice. In contrast, behavioral parenting (i.e., SafeCare) 
offers a very structured curriculum, training in a few, specific parenting skills that are seen as 
key to improving outcomes, and includes less of an explicit focus on emotional support as a 
primary service goal.   
It is common practice for consumers to rate service satisfaction at the completion of services. 
However, ratings are typically made at the end of service, and thus (1) only clients who reach the 
end of services are able to offer ratings (dropouts do not), and (2) there is little understanding of 
client perception of services throughout the course of treatment.  The goal of this aim is therefore 
to understand client reactions to the two interventions (SafeCare and SCM) offered in this trial 
across the course of treatment.  We will employ mixed methods (both quantitative and 
qualitative data collection) to inform this question. Specifically, we will (1) conduct qualitative 
interviews with families at two time points during the course of service, and (2) collect session-
by-session ratings from families on service reaction (perceived burden satisfaction, perceived 
effectiveness).  
Aim 3. Compare SC to SCM on parent outcomes and child outcomes including parenting skill, 
parenting stress, and parent-child relationship, and child behavioral, social and emotional 
well-being 
Child welfare systems have a legislative mandate to improve three aspects of child and family 
functioning: safety, permanency and well-being. Most research on these topics has focused on 
the outcomes of safety (cases of re-abuse) and permanence (child removals from the home).  The 
third aspect of the child welfare system’s mandate – well-being – has received much less 
attention with respect to intervention effectiveness, and client’s preferences about important 
outcomes.   
As conceptualized in the original proposal, there is a strong need to assess the impact of child 
welfare intervention on variables that are of importance to consumers in the child welfare system 
(parents). Here we will compare two interventions – SafeCare and SCM – on a number of 
aspects of parent and child well-being: parenting variables, parent mental health, and child 
behavioral, social, and emotional well-being.  By focusing on measures of well-being and by 
engaging consumers in defining this focus, this study holds potential to expand the way in which 
the program effectiveness is conceptualized.  This could be particularly important for 
dissemination of parenting programs to at-risk parents, who may be interested in different 
outcomes (e.g., improving their child’s behavior) than child welfare systems, and findings may 
be useful for making interventions more appealing to consumers.  
 
C. METHOD 
C1. Site Recruitment  
Several recruitment strategies will be employed.  One-page information sheets will be distributed 
describing the purpose of the project and information about The National SafeCare Training and 

8 
 
Research Center (NSTRC), as well as site requirements and eligibility information. We will 
distribute information through personal contacts of the investigative team and a variety of email 
lists. Email lists targeted include:  
 NSTRC’s listserve (contains current and prospective sites) 
 Child Maltreatment Researcher list (hosted by National Data Archive on Child Abuse and 
Neglect at Cornell University)   
 American Psychological Association - Division 37 (Society for Child and Family Policy 
and Practice) 
 The National Center for Community-Based Child Abuse Prevention   
 ACF’s listserve of Discretionary grantee  
 FRIENDS, the National Center for Community-Based Child Abuse Prevention 
Once we begin receiving inquiries about the study, we will organize a webinar to present project 
details which allows interested agencies to ask questions.  Following our initial webinar, we will 
prepare for a second wave of recruitment.  We will aim to target individuals involved in child 
welfare by sending information, including our flyer and recorded webinar, to Community-Based 
Child Abuse Prevention lead agency contacts across the country, as well as child welfare 
directors.  During our third wave of recruitment, we will research child welfare agencies across 
the country and develop an extensive list of county and regional child welfare contacts.  Each 
contact will be sent a personalized email explaining the opportunity, and how to proceed if 
interested.           
C2. Site Application and selection  
As sites express interest, we will engage them in our screening process, which will determine 
whether or not pursuing an application would be appropriate.  Our initial conversations with 
interested sites will focus on team structure concerning randomization, number of referrals and 
their source, as well as whether or not the cases of maltreatment are substantiated.  Upon 
successful completion of the screening process, potential sites will be required to complete our 
application.  Completed applications will then be reviewed by our review team, comprised of a 
selection of research team members, partners, and stakeholders.  Conference calls will be held 
with each site to clarify points of confusion, for example, what services are currently being 
conducted, specific target populations, and if and how randomization could be accomplished.  
Once applications are approved, we will begin readiness conversations with potential sites.  
During several of these calls we will discuss the randomization piece.  Prior to the overview 
webinar, team randomization will be conducted.    
C3. Interventions  
C3.1 
Supportive Case Management  
SCM is a loosely defined, process oriented intervention in which providers engage families 
through relationship building and emotional support, assessing needs and linking families to 
local services, as well as troubleshooting and addressing day-to-day crises. SCM providers visit 
the home weekly to provide services.  They do not have specific goals in mind for each particular 
session, but determine what needs to be addressed with the family in real time.  There will likely 
be variability in SCM delivered at each site in terms of length of service and frequency of visits.  
For all clients receiving SCM, we will document number and frequency of visits, referrals, and 

9 
 
ancillary services utilized.  We will incorporate these variables into the analyses of heterogeneity 
of treatment effects described below.  We expect that SafeCare and SCM will be of similar dose 
and contact frequency, given that these dimensions drive CBO reimbursement.  To the extent 
possible, we will encourage this equivalence within each site. 
C3.2 
SafeCare 
SafeCare is a highly structured, empirically-supported parenting program that addresses the 
proximal behaviors that can lead to child neglect and physical abuse (Hecht, Silovsky, Chaffin, 
& Lutzker, 2008; Lutzker & Bigelow, 2002; Whitaker, Lutzker, Self-Brown, & Edwards, 2008). 
SafeCare was developed to help parents involved with the child welfare system, or who are 
members of high-risk populations. SafeCare is delivered weekly in the natural environment, 
usually the home, over an 18-20 week period. SafeCare contains three modules – health, safety, 
and parenting – that all families receive, and SafeCare providers utilize structured problem-
solving on an as needed basis.   
The health module uses standardized, validated scenarios to teach parents skills to care for their 
children’s health. Parents are taught to recognize symptoms of illness and injury and to use a 
decision making process to choose the appropriate care for the child.  The safety module aims to 
make homes safer and healthier for children while promoting parental supervision. In the safety 
module, parents are taught about ten categories of home hazards, and then learn to make the 
home safe by eliminating or securing the hazard.  The parent-child interaction (PCI) module 
promotes positive relationships and limit setting by promoting positive parent-child interactions 
and preventing problem situations that can lead to harsh parenting. PCI teaches parents to 
increase positive interactions (talking, affectionate touching) with the child through play and 
daily interactions, and also how to manage the child’s behavior using behavioral techniques, 
structuring interactions, and reinforcing positive behaviors.  Parents of infants focus on behaviors 
that promote bonding (e.g.., looking, holding, talking, touching) versus behavior management.  
The NSTRC is the main purveyor of the SafeCare program, along with other University-based 
sites, and provides training and implementation support to sites that choose to adopt SafeCare.  
Importantly, NSTRC employs a rigorous training and support model to ensure adoption and 
implementation of the program model with fidelity, which is needed to replicate outcomes. 
Implementation of SafeCare  
Once a site is selected, all providers and agency leadership are invited to attend a webinar 
describing the study, their involvement, and are given the opportunity to ask questions about the 
project.  In addition to the project overview webinar, the in-person orientation is designed to 
address any questions or concerns expressed by agency staff and/or leadership.  The orientation 
typically takes place two weeks before implementation and is led by the faculty member 
assigned to the site and the lead trainer.    
Teams randomized to SafeCare will be assigned training specialists who will conduct the 4-day 
workshop training.  The GSU research team will work with sites to schedule training and ensure 
that staff are prepared for implementation.   
Due to the various locations of sites across the country, the GSU research team will request sites 
to determine which gift card should be distributed to families participating in the project at their 
site.    

10 
 
C3.3 
Tracking interventions received and utilization 
We will capture basic service utilization data from providers to track client use of services. Data 
will be tracked with a smartphone/tablet app that is made available to all providers.  The app 
allows providers to enter a new session, enter services offered, and for SafeCare clients, track 
behavior change.  The app will provide information for each client on number of sessions 
(including dates), program completion, and types of services offered. SafeCare sessions are 
scripted and thus the content delivered is very clear and consistent across sessions. Sessions 
delivered in the SCM intervention will include a variety of content.  For non-SafeCare sessions, 
providers will indicate from a list of activities what was done during the session. They will 
include the number of minutes spent on each activity, in order to describe the overall services 
delivered across sessions. We will also collect information on additional service referrals 
provided to parents and whether or not they attended additional programs/services.   
C4. Aim 1. Identify and assess barriers and facilitators that influence implementation of a 
structured behavioral parenting program 
C4.1 
Overview  
The approach to this aim will be primarily a qualitative approach with individual semi-structured 
interviews and focus groups conducted with staff at the sites. We will invite staff at all sites to 
participate, including Montgomery County MD, who dropped out immediately after training.  
We will recruit all administrators and supervisors to participate in individual interviews. For 
providers, we will conduct focus groups at each site with the SafeCare teams (10 focus groups; 
one at each implementing site plus Montgomery County) and non-SafeCare teams (9 focus 
groups; one at each implementing site).  
The team, including both researchers and partners, will jointly develop a semi-structured 
interview and focus group guide. Dr. Cathleen Willging will lead the effort as the team’s 
qualitative research expert; participation will also include the current members of the Georgia 
State University (GSU) research team (e.g., Whitaker and Self-Brown), and partners representing 
the perspectives of administrators (e.g., Shoecraft, Neighbors-Day), agency 
supervisors/providers (Perez), and parents (Still). Development of the interview will be an 
iterative process in which the team will brainstorm domains of interest. Willging will then 
develop specific questions and pass them on to the team for review and revision.  
Several domains of initial interest will be drawn from the implementation science literature. We 
will consult the EPIS conceptual model to spell out specific concepts to explore in the interviews 
and focus groups.  Initial domains we will target are:  
 Funding agency role in implementation 
 Engagement of implementation developers  
 Organizational leadership and alignment across staffing levels  
 Organizational structure, culture, and climate  
 Fit of service model (SC) with client needs  
 Time pressures/caseload issues/competition demands 

11 
 
 Individual attitudes, beliefs, and expectations about services  
Data collection will be conducted entirely by Dr. Willging’s team.  This is important because 
Willging and her team are not part of the NSTRC, and this will reduce the pressure on 
participants to respond favorably regarding the SafeCare intervention. It is possible there are 
“fit” issues between SafeCare and the client needs, and provider must feel free to voice those 
concerns.  
C4.2 
Qualitative 
Provider, supervisor, and administrative recruitment  
Providers, supervisors, and administrators will be informed of the opportunity to participate in 
qualitative interviews over a series of conference calls and webinars.  Those interested will be 
asked to sign a consent form and work with the GSU research team for scheduling purposes.       
Assessment Procedures  
We will conduct focus groups at each site with the SafeCare teams (10 focus groups; one at each 
implementing site plus Montgomery County) and non-SafeCare teams (9 focus groups; one at 
each implementing site). A total of 19 focus groups (n = 4-8 providers in each) and up to 45 
individual interviews (average of 5 per site) will be conducted.  The GSU-based research team 
will coordinate the recruitment of participants and the scheduling of the interviews. Dr. Willging 
will train research team members (Masters- or BA-level staff) to conduct focus groups and 
qualitative interviews. Team members will travel to each of the sites to conduct focus groups and 
individual interviews. If all individual interviews cannot be conducted at the same trip as focus 
groups, we will conduct remaining interviews via phone or video chat.    
Data Analysis 
Data will be transcribed and analyzed as follows.  First, open coding will be conducted to locate 
themes/issues. Codes will be assigned to segments of text ranging from a phrase to several 
paragraphs based a priori on topical domains and questions from the interview guides. New 
codes not previously considered will be identified and defined. Second, “focused coding” will be 
used to determine which of these themes/issues emerge frequently and which represent unusual 
or particular concerns to the participants. All qualitative data will be independently coded by at 
least two members of the research team, and inter-rater reliability tested against a Kappa 
benchmark.   
 
C5. Aim 2. Understand parent and provider reactions to SC and SCM services, especially 
parent perceptions related to trajectory of burden, satisfaction, and perceived impact 
across intervention receipt 
We will assess both family and provider response to the interventions.  For family responses, we 
will employ mixed methods for assessing family responses collecting data via both qualitative 
interviews and survey data. For provider data, we will rely primarily on the quantitative surveys 
collected periodically throughout services. Baseline data for families and providers will allow us 
to examine potential effect moderators.  
C5.1 
Family burden, satisfaction and perceived impact 

12 
 
For the qualitative component  with this study, we will conduct semi-structured interviews with 
72 families (36 SC, 36 SCM) at two time points, the first at approximately 4-5 sessions (of 18-
20) into service, and then again at the end of service.  The selection of four sessions into service 
as the first assessment point was chosen to allow parents to get a sense of the services that will 
be provided, and thus represents their initial impressions of services.  
We will also collect sessions-by-sessions data from all families enrolled in the study. Providers 
will use a smartphone app that is provided as part of the study to collect data on quantitative 
measures.  At the end of each session, families will be asked to respond to 10 questions via the 
app that assess their perceptions of burden, satisfaction, and service effectiveness.  Once a parent 
submits responses, the app locks the survey and thus providers cannot access parent responses.  
The results of the quantitative and qualitative data for this Aim will be integrated by the 
research/partner team. Specifically, we will compare findings from the quantitative and 
qualitative analyses, and specifically to look for convergences and divergences in findings (do 
both methods similar or different findings?), and we will use the qualitative responses to expand 
on the quantitative findings to provide deeper meaning (Palinkas et al., 2011).  For example, we 
may find in both sets of data that burden, satisfaction, and service effectiveness change over 
time. The qualitative data will also be used to inform the quantitative data on specific of burden, 
satisfaction, and effectiveness. The quantitative data assess the extent of burden, satisfaction, and 
effectiveness, but the qualitative data will be used to assess why (or why not)?  
Quantitative assessments of burden, satisfaction, and program effectiveness  
Family perceptions of service burden, satisfaction, and program effectiveness will be captured in 
two ways: (1) during receipt of service through quantitative data collection and, (2) post-service 
through qualitative interviews from a select number of participants.   
After each session, participants will complete a brief 10-item survey that assesses perceived 
service burden, satisfaction, and program effectiveness.  Participants will complete the survey on 
a smartphone or tablet that the provider will bring to each session (the provider will use it to 
collect service data). The specific questions were developed for this study as there has been little 
assessment of these constructs in a child welfare setting.  Questions are included in the 
Appendix.  
Qualitative interviews to assess burden, satisfaction, and effectiveness 
A subset of participants in the research study (n=72; 36 who received SafeCare and 36 who 
received SCM) will be asked of their interest in participating in the qualitative portion of the 
study, consisting of in-depth interviews via a separate consent process.  The interviews will be 
conducted with parents at two time points, the first at approximately 4-5 sessions (of 18-20) into 
service, and then again at the end of service. Interviews will focus on how parents experienced 
the intervention, in addition to their initial expectations and perceived short-term outcomes. 
Eight families from each site, 72 total, will be included in the qualitative study. We will 
purposely choose families representing diversity in gender, race/ethnicity, and other 
demographics to capture a full range of perspectives.   
Interview guides comprised of approximately 10 to 12 open-ended questions will be developed 
based on input from our provider and patient partners, pilot tested, and then revised.  The guide 
will be designed to assess key issues that bear upon implementation (e.g., characteristics of the 

13 
 
model, modules, or provider behaviors found most or least helpful) and will include follow-up 
probes. Interviewers will be trained to examine novel or emergent issues that were not 
considered previously but which arose in the course of their discussions with parents.  All 
interviews will be recorded and professionally transcribed with a final accuracy review 
undertaken by the interviewer, who will also take detailed notes during each interview.  These 
notes will be organized according to a standard format (or “debriefing form”), with information 
on date, time, length of the interaction, physical setting, and preliminary themes.(Miles, 
Huberman, & Saldaña, 2013) The typed notes and transcripts will be converted into analyzable 
text and imported into a password protected NVivo 10 database for organization and analysis via 
iterative readings or codings. Discussion of preliminary themes during team meetings will lead to 
revisions of the interview guide.  Parents will be compensated $25 for their time participating in 
each interview. 
C5.2 
Provider burden, satisfaction and perceived impact   
Providers who consent to the study will be surveyed at baseline (prior to implementation), then 
quarterly over the course of the next year, and will complete a final survey one year post-
baseline.  
The initial (baseline) and final surveys are comprehensive, including measures of demographics, 
work experience, organizational factors (culture, climate, leadership) and individual attitudes and 
beliefs, that may affect implementation and service quality.  The quarterly surveys are brief and 
focus on service implementation, burden, quality, and perceived effectiveness.  Thus, we will 
have longitudinal survey data from providers at four time points during implementation 
regarding intervention burden, satisfaction, and perceived effectiveness.  
All providers were recruited to participate in surveys conducted at the beginning and end of 
implementation.  Providers willing to participate will sign a consent form, complete two 
organizational surveys (one prior to implementation and one at the end), and the  brief quarterly 
surveys assessing program satisfaction, burden, and perceived effectiveness of the intervention 
they are delivering (SCM or SafeCare).   
Providers who consent to the study are surveyed at baseline (prior to implementation), then three 
times quarterly over the course of the next year, and complete a final survey one year post-
baseline. The initial (baseline) and final surveys are comprehensive, including measures of 
demographics, work experience, organizational factors (culture, climate, leadership) and 
individual attitudes and beliefs, that may affect implementation and service quality.  The 
quarterly surveys are brief and focus on service implementation, burden, quality, and perceived 
effectiveness.  All provider data will be collected using web-based surveys developed through 
Qualtrics. The GSU-based project coordinator will track providers who consent to the study. 
Providers will be emailed and called if necessary when it is time to complete the next survey.  
Providers who leave the organization before the end of the 12-month period will be asked to 
complete the follow-up survey of organization and individual characteristics at the time of their 
departure.   
Analyses of provider responses will be done using GLMM as responses are nested within 
providers.  We will have up to five responses for providers.  Analyses will focus on 
understanding mean differences between SC and SCM providers, as well as time trends, and 
whether differences follow any time trends. 

14 
 
 
C6. Aim 3. Compare SC to SCM on parent outcomes and child outcomes including 
parenting skill, parenting stress, and parent-child relationship, and child behavioral, 
social and emotional well-being 
Families will complete self-report measures at two time points (baseline and 6- months) to assess 
major outcomes. Assessments will be done via tablet-based surveys in the parents’ home 
conducted by blinded assessors.   
C6.1 
Target population  
The target population for this aim are families (caregivers/parents and children) who are being 
served by child protective service systems or their contractors for maltreatment. We will include 
both families who have cases of maltreatment that are substantiated and those who are not 
substantiated but referred to CPS systems for possible maltreatment.  Research has shown 
identical risk profiles and likelihood of recidivism for CPS-referred families regardless of 
whether a referral is formally substantiated (Kohl, Jonson-Reid, & Drake, 2009), and the two 
populations are typically served with identical interventions.   
A total of 376 parents/ caregivers will be recruited by the end of the study based on our current 
recruitment rate of families recruited per provider per month.  
C6.2 
Family Recruitment 
The research team will work with local service providers to gain family interest in the project. 
All families enrolled in the study will already be receiving services from providers.  For the 
study, we will ask providers to briefly describe the project, including that it is completely 
voluntary, and request permission to pass the family contact information to the GSU-based 
project coordinator.  The GSU team will contact interested participants, fully explain the study 
procedures, and review the consent form with interested families.  If a family consents to the 
project, the GSU-based research coordinators will contact a local assessor and provide the family 
name and phone number.  An assessor, who will be a GSU contract-based employee and blinded 
to study conditions will be sent to conduct the face to face assessment.  Assessments will be done 
in the home and will consist of an electronic survey and a home safety assessment. The survey 
will use standardized questionnaires delivered to clients on a tablet. The local assessor will 
schedule the assessment prior to the next visit by the service provider. 
C6.3 
Family assessments of outcomes  
Families will complete in-home assessments at baseline and 6-months.  Each assessment will 
consist of a computer-assisted self-interview and a home safety scan to be conducted by the 
assessor.  
All constructs identified below will be assessed via self-reported survey conducted on tablet, 
except for the assessment of home hazards, which will be assessed via the safety scan. Parents 
will be compensated with a $40 gift card for each assessment completed. 
Local data collectors will be hired and trained at each site to conduct family assessments, and 
will be paid on a contractual basis for each assessment.  The GSU research team will work with 
sites to advertise and identify candidates for positions. Appropriate candidates will be naive to 
the intervention, hired by GSU directly via contract mechanism, and trained at their site by the 

15 
 
GSU research coordinator.  GSU research staff will conduct a 1-day, in-person training for data 
collectors.  Data collectors will be trained in scheduling, assessment, administrative, and follow-
up procedures.  They will be provided with a tablet for the duration of the study and will be 
trained on how to conduct in-home assessments.  Local assessors will be required to complete 
CITI training and will be part of GSU’s IRB application.   
Computer assisted self-interview 
Parents/caregivers will complete a survey on a tablet provided by the assessor. The survey will 
be developed with the Qualtrics program. The survey will use standardized questionnaires 
delivered to clients on a tablet.  Parents will complete the electronic survey independent of the 
assessor to maintain privacy, while the assessor helps watch over children in the home to provide 
the parent additional uninterrupted time.  If a parent needs assistance reading, the assessor will 
read the question aloud to the parent (using paper measures) and the parent will enter their 
responses directly into the tablet.  The specific constructs to be measured include the following:  
 Parenting Behaviors – the Parenting Young Children Scale (McEachern et al., 2012) assesses 
three dimensions of positive parenting: limit setting, proactive parenting, and supporting 
positive behavior.   
 Parent-child Relationship/Bonding/Attachment –  the quality of the relationship between the 
parent and child will be measured with the attachment subscale of the Devereaux Early Child 
Assessment (LeBuffe & Naglieri, 1999). 
 Parenting Stress –  the Parenting Stress Inventory – short form (Abidin, 1995) is a 36-item 
scale designed to measure stressors in parenthood including parental distress, dysfunctional 
interactions, and stressors related to having a difficult child.   
 Parent Emotional Well-being – the Brief Symptom Inventory (Derogatis & Melisaratos, 
1983) is a 53-item scale designed to measure a range of emotional health states including 
depression, anxiety, somatization, and others. We will also use the SF-12, (Gandek et al., 
1998) a brief 12-item scale assessing physical and emotional health.  
 Child Well-being – the Devereaux Early Child Assessment (DECA) (LeBuffe & Naglieri, 
1999) is a validated behavioral rating scale that assesses child behavior in the domains of 
initiative, self-control, and behavioral concerns.  
 Caregiving Behaviors – the Mother-Child Neglect Scale (MCNS) (Lounds, Borkowski, & 
Whitman, 2004) is a 20-item scale designed to assess caregiving behaviors in four domains: 
physical, cognitive, supervision, and emotional needs.   
 Home Environment – the CHAOS scale (Confusion, Hubbub, and Order) (Dumas et al., 
2005) is a 15-item scale that will be used to measure structure and chaos in the home 
environment, a common well-being related goal among parents. 
 Family Resources – the Family Resources Scale – Revised (Van Horn, Bellis, & Snyder, 
2001) assesses the adequacy of family needs in four domains: basic needs, money, time for 
self, time for family. Resource sufficiency is a common facet of parent well-being.  
 Drug/Alcohol Use – the ASSIST (Humeniuk et al., 2008) measures drug and alcohol use and 
drug/alcohol related problems. 

16 
 
 Relationship Violence - the short form of the Conflict Tactics Scale II(Straus & Douglas, 
2004) will be used to measure relationship violence victimization and perpetration.   
 Social Support – a subscale of the Protective Factors Survey (Counts, Buffington, Chang-
Rios, Rasmussen, & Preacher, 2010) will measure concrete and emotional support, each 
using a five item scale.  
 Standard Demographics – standard demographics from the parent will be collected including 
age, sex, education, income, and marital status. We will also collect information about family 
structure, number of children, living situation, relationship partners, and prior social and 
mental health services.  
 Parental Supervision and Childhood Injury – a questionnaire adapted for the current study 
which assess supervision in the home and injuries from hazards in the home  
Home Safety Scan 
Assessors will conduct a home safety scan designed to assess the hazards in the home. In the 
safety scan, assessors will look in three rooms in the house – the kitchen, bathroom, and living 
room – and count the number of accessible hazards in those rooms, using guidelines defined in 
the Home Accident Prevention Inventory-Revised.58 The home safety scan is included in the 
Appendix.  
C7. Data Management    
All data cleaning and analysis will be handled at Georgia State University. Assessment data from 
families and providers will be collected via web-based surveys, and downloaded by the GSU 
research team, so no additional data transfer will be needed.  Data on service utilization collected 
via tablets will be stored on a secure cloud-based server controlled by the GSU research team.  
The GSU research team will enter, clean, and analyze the data.  Oversight of data entry and 
management will be done by the PI (Whitaker) and Self-Brown.  Dr. Hayat (biostatistician) will 
oversee the data cleaning and analysis.  This arrangement is purposeful and will allow the study 
analysts to maintain independence of the purveyors of SafeCare at NSTRC (Whitaker, Self-
Brown, Lutzker). 
C8. Data cleaning and missing data 
Prior to analysis, data will be cleaned and inspected for missing data.  Statistical analyses will be 
handled in a way that takes advantage of all available observed data (i.e. does not discard any 
cases) while not over-estimating precision.  Missing data will be assumed to be missing at 
random (MAR). Although there is no complete, comprehensive method for handling missing 
data for general linear models, there are several approaches that we can draw upon to address 
this issue (Diggle, Heagerty, Liang, & Zeger, 2002).  We will use multiple imputation of missing 
data whereby missing observations for individuals are estimated based on other covariates in the 
data set. If initial examination of the data and field reports suggest that the MAR assumption is 
not supported, we will conduct sensitivity analysis under the missing not at random (MNAR) 
assumption using pattern mixture models. However, the availability of only three longitudinal 
data points will limit the extent to which this is possible.   
C9. Statistical power for quantitative analyses 

17 
 
In the initial proposal, we identified several standardized effects from prior studies for parenting 
skill (Carta, Lefever, Bigelow, Borkowski, & Warren, 2013) d = .81, parental depression 
(Chaffin, Bard, Bigfoot, & Maher, 2012) d = .33, changes in children’s behavior (Carta et al., 
2013) d = .29, and parent satisfaction (Damashek, Bard, & Hecht, 2012; Damashek, Doughty, 
Ware, & Silovsky, 2011; Silovsky et al., 2011) d = .52.   Though there is variability in these 
effect, the median effect appears to be a medium-sized  effect (Cohen, 1988).  
To compute power for nested data, we used the Optimal Design program (Sprybrook et al., 2011) 
which accounts for the nested structure of the data.  In the initial proposal, we used provider as 
the nesting unit, with respondents/parents nested within providers, and time nested within 
respondent/parent.   
We have generated power estimates based on each method of projecting sample size.  The first 
method, the simple average of clients recruited per month, suggests a sample size of 144 families 
across the 23 teams; this would include 1 family per provider or about 6 families per team. The 
second method, using provider-months, suggests a sample size of 376 providers recruited across 
the 27 teams; this would include about 2 families per provider, or about 14 families per team.  
For Aim 2, understanding parent reactions to SC and SCM services, analyses are based on 
parents’ session-by-session responses taken immediately after each intervention session. Each 
respondent should have as many responses as sessions (up to 20). For sample size determination, 
we have used a modest number of repeated measures – 10 – because some families may 
terminate services early. For a sample size of 376 families across 27 teams, with alpha set to .05, 
and power at .80, we are able to detect effects sized d = .31 to d = .45 for ICC values ranging 
from .01 to .10.  For a sample of 144 families across 23 teams, under the same assumptions, 
minimally detectable effect sizes are d = .51 to d = .62.  
For Aim 2, assessing provider perceptions of burden, satisfaction, and effectiveness, providers 
are the unit of analysis and measurement is repeated over four time points.  To date, we have 
recruited 81% of providers into the study, and this projects to 139 providers over the total 
provider sample size of 172. To compute statistical power, we used G*Power (Faul, Erdfelder, 
Lang, & Buchner, 2007) to compute minimally detectable effect size given the sample of 139 
providers with four assessment points, and assuming alpha = .05 and power = .80. For a sample 
of 139 providers, power is adequate to detect a small sized effect of d = .20, and if we assume 
attrition to be 20% (n = 111 providers), the detectable effect size is a small sized effect of d = 
.22.  
For Aim 3, detecting differences in parenting skill, parent mental health and well-being, and 
child behavioral, social, and emotional development, the power analysis is based on 23 teams of 
providers, with each team providing family data on 14 families for the estimate using provider 
months to project enrollees (n = 376) and 6 families for the estimate using the average number of 
families enrolled per month to date (n =144).  For the former estimate (27 teams; 14 families per 
team, n ~ 376), with a level of significance of .05, the cluster randomized design with two 
repeated measures, and power at .80, we are able to detect effect sizes in the range of d = .38 to d 
= .50 for values of ICC ranging from .01 to .10.  For the latter estimate (23 teams; ~6 families 
per team, n = 144), with a level of significance of .05, the cluster randomized design with 
repeated measures provides power at 0.80 to detect effect sizes in the range of d = .62 to d = .70 
for values of ICC ranging from .01 to .10.  Thus, the former method of projecting sample size 

18 
 
appears to yield sufficient power to justify the study (medium sized effects), while the latter 
perhaps does not (greater than medium-sized effects).   
C10. Ethical Considerations 
We believe the risks of study participation to be very minimal. All survey data will be collected 
via web-based data collection. Web-based data will be collected using www.qualtrics.com, a 
secure online data collection system for surveys and psychological research, and through the 
SafeCare app. Responses obtained from participants will be collected only for the purposes of 
this research and this information will be kept strictly confidential to guarantee the protection of 
human subjects. Specifically, data will be kept secure using standard methods. Only the research 
team will have access to the data. Only code numbers will be used on data forms and in 
electronic databases and no identifying information will be associated with participants’ 
responses. Responses obtained from the research study will be secured on a secured network 
drive or in a locked filing cabinet in a locked office. No individual data will be released. 
 
 

19 
 
D. REFERENCES  
 
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. C. (2011). Advancing a conceptual model of 
evidence-based practice implementation in public service sectors. Administration and 
Policy in Mental Health and Mental Health Services Research, 38, 4-23.  
Abidin, R. R. (1995). Parenting Stress Index: Professional Manual. Lutz, FL: Psychological 
Assessment Resources. 
Carta, J. J., Lefever, J., B., Bigelow, K., Borkowski, J., & Warren, S. F. (2013). Randomized 
Trial of a Cellular Phone-Enhanced Home Visitation Parenting Intervention. Pediatrics, 
132(Supplement 2), S167-S173. doi:10.1542/peds.2013-1021Q 
Chaffin, M., Bard, D., Bigfoot, D. S., & Maher, E. J. (2012). Is a Structured, Manualized, 
Evidence-Based Treatment Protocol Culturally Competent and Equivalently Effective 
Among American Indian Parents in Child Welfare? Child Maltreatment, 17(3), 242-252.  
Chaffin, M., Funderburk, B., Bard, D., Valle, L. A., & Gurwitch, R. (2011). A combined 
motivation and parent–child interaction therapy package reduces child welfare recidivism 
in a randomized dismantling field trial. Journal of Consulting and Clinical Psychology, 
79(1), 84-95.  
Chaffin, M., Hecht, D., Bard, D., Silovsky, J. F., & Beasley, W. H. (2012). A Statewide Trial of 
the SafeCare Home-Based Services Model with Parents in Child Protective Services 
Pediatrics, 129, 509-515.  
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: 
Lawrence Erlbaum Associates. 
Counts, J. M., Buffington, E. S., Chang-Rios, K., Rasmussen, H. N., & Preacher, K. J. (2010). 
The development and validation of the protective factors survey: A self-report measure of 
protective factors against child maltreatment. Child abuse & neglect, 34(10), 762-772.  
Damashek, A., Bard, D., & Hecht, D. (2012). Provider Cultural Competency, Client Satisfaction, 
and Engagement in Home-Based Programs to Treat Child Abuse and Neglect. Child 
Maltreatment, 17(1), 56-66.  
Damashek, A., Doughty, D., Ware, L., & Silovsky, J. (2011). Predictors of Client Engagement 
and Attrition in Home-Based Child Maltreatment Prevention Services. Child 
Maltreatment, 16(1), 9-20.  
Derogatis, L. R., & Melisaratos, N. (1983). The brief symptom inventory: an introductory report. 
Psychological medicine, 13(03), 595-605.  
Diggle, P., Heagerty, P., Liang, K.-Y., & Zeger, S. (2002). Analysis of longitudinal data: Oxford 
University Press. 

20 
 
Dumas, J. E., Nissley, J., Nordstrom, A., Smith, E. P., Prinz, R. J., & Levine, D. W. (2005). 
Home chaos: sociodemographic, parenting, interactional, and child correlates. J Clin 
Child Adolesc Psychol, 34(1), 93-104. doi:10.1207/s15374424jccp3401_9 
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical 
power analysis program for the social, behavioral, and biomedical sciences. Behavior 
Research Methods, 39, 175-191.  
Felitti, V. J., Anda, R. F., Nordenberg, D., Williamson, D. F., Spitz, A. M., Edwards, V., . . . 
Marks, J. S. (1998). Relationship of Childhood Abuse and Household Dysfunction to 
Many of the Leading Causes of Death in Adults The Adverse Childhood Experiences 
(ACE) Study. American Journal of Preventive Medicine, 14(4), 245-258.  
Fergusson, D. M., Boden, J. M., & Horwood, L. J. (2008). Exposure to childhood sexual and 
physical abuse and adjustment in early adulthood. Child abuse & neglect, 32(6), 607-619.  
Gandek, B., Ware, J. E., Aaronson, N. K., Apolone, G., Bjorner, J. B., Brazier, J. E., . . . Prieto, 
L. (1998). Cross-validation of item selection and scoring for the SF-12 Health Survey in 
nine countries: results from the IQOLA Project. Journal of clinical epidemiology, 51(11), 
1171-1178.  
Gilbert, R., Widom, C. S., Browne, K., Fergusson, D., Webb, E., & Janson, S. (2009). Burden 
and consequences of child maltreatment in high-income countries. The Lancet, 
373(9657), 68-81.  
Hecht, D. B., Silovsky, J. F., Chaffin, M., & Lutzker, J. R. (2008). SafeCare: An evidence-based 
approach to prevent child neglect. The APSAC Adviser, Winter, 14 - 17.  
Herrenkohl, E. C., Herrenkohl, R. C., Rupert, L. J., Egolf, B. P., & Lutz, J. G. (1995). Risk 
factors for behavioral dysfunction: the relative impact of maltreatment, SES, physical 
health problems, cognitive ability, and quality of parent-child interaction. Child abuse & 
neglect, 19(2), 191-203.  
Humeniuk, R., Ali, R., Babor, T. F., Farrell, M., Formigoni, M. L., Jittiwutikarn, J., . . . 
Monteiro, M. (2008). Validation of the alcohol, smoking and substance involvement 
screening test (ASSIST). Addiction, 103(6), 1039-1047.  
Kohl, P. L., Jonson-Reid, M., & Drake, B. (2009). Time to Leave Substantiation Behind: 
Findings From A National Probability Study. Child Maltreatment, 14(1), 17.  
LeBuffe, P. A., & Naglieri, J. A. (1999). The Devereux early childhood assessment. Lewisville, 
NC: Kaplan Press Publishing.  
Lounds, J. J., Borkowski, J. G., & Whitman, T. L. (2004). Reliability and Validity of the Mother-
Child Neglect Scale. Child Maltreatment, 9(4), 371.  
Lutzker, J. R., & Bigelow, K. M. (2002). Reducing Child Maltreatment: A Guidebook for Parent 
Services. New York, NY: Guilford Publications. 

21 
 
McEachern, A. D., Dishion, T. J., Weaver, C. M., Shaw, D. S., Wilson, M. N., & Gardner, F. 
(2012). Parenting Young Children (PARYC): Validation of a self-report parenting 
measure. Journal of Child and Family Studies, 21(3), 498-511.  
Miles, M. B., Huberman, A. M., & Saldaña, J. (2013). Qualitative data analysis: A methods 
sourcebook: SAGE Publications, Incorporated. 
Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. 
(2011). Mixed method designs in implementation research. Administration and Policy in 
Mental Health and Mental Health Services Research, 38(1), 44-53.  
Prinz, R. J., Sanders, M. R., Shapiro, C. J., Whitaker, D. J., & Lutzker, J. R. (2009). Population-
based prevention of child maltreatment: The US Triple P system population trial. 
Prevention Science, 10(1), 1-12.  
Silovsky, J., Bard, D., Chaffin, M., Hecht, D. B., Burris, L., Owora, A., . . . Lutzker, J. (2011). 
Prevention of child maltreatment in high risk rural families: A randomized clinical trial 
with child welfare outcomes. Children and Youth Services Review, 33(8), 1435-1444.  
Sprybrook, J., Bloom, H., Congdon, R., Hill, C., Martinez, A., & Raudenbush, S. W. (2011). 
Optimal Design Plus Empirical Evidence: Documentation for the “Optimal Design” 
Software. http://hlmsoft.net/od/. 
Straus, M. A., & Douglas, E. M. (2004). A short form of the Revised Conflict Tactics Scales, and 
typologies for severity and mutuality. Violence Vict, 19(5), 507-520.  
Thornberry, T. P., Ireland, T. O., & Smith, C. A. (2001). The importance of timing: The varying 
impact of childhood and adolescent maltreatment on multiple problem outcomes. 
Development and psychopathology, 13(04), 957-979.  
Van Horn, M. L., Bellis, J. M., & Snyder, S. W. (2001). Family Resource Scale-Revised: 
Psychometrics and validation of a measure of family resources in a sample of low-
income families. Journal of Psychoeducational Assessment, 19(1), 54-68.  
Wegman, H. L., & Stetler, C. (2009). A meta-analytic review of the effects of childhood abuse 
on medical outcomes in adulthood. Psychosomatic Medicine, 71(8), 805-812.  
Whitaker, D. J., Lutzker, J. R., Self-Brown, S., & Edwards, A. E. (2008). Implementing an 
evidence-based program for the prevention of child maltreatment: The SafeCare® 
program. Report on Emotional & Behavioral Disorders in Youth, 8, 55-62.