Parent Training in Pediatric Care: A Self Directed Tablet-Based Approach
Phase: NA
Conditions: Parent Training, Primary Care
Sponsor: Rush University Medical Center

Study Protocol and Statistical Analysis Plan  
 
Identifiers: 
NCT02723916    
Title: Parent Training in Pediatric Care: A Self Directed Tablet-Based Approach 
 
Date of Document: September 1, 2015 
 
 

The specific aims are: 
Aim 1. Test the direct effects of the 6-module eCPP on parenting outcomes (parenting behavior, self-efficacy, 
and stress) and child outcomes (child problem and prosocial behavior) compared to an enhanced usual 
care control condition among low-income parents with young children seen in primary care.  
HYPOTHESIS 1A: Relative to parents in the control condition, parents in the eCPP condition will report 
greater improvements in parenting and child outcomes at the 3-month assessment.  
HYPOTHESIS 1B: The improvements in parenting outcomes and child outcomes the eCPP condition relative 
to the control condition will be maintained through the 6- and12-month assessments.  
Aim 2. Compare the cost-effectiveness of the eCPP relative to control for parenting and child outcomes. 
HYPOTHESIS 2: The eCPP is cost-effective relative to control for parenting and child outcomes. 
Aim 3. Quantify the levels of program implementation of the eCPP in primary care using the RE-AIM 
framework. RESEARCH QUESTION 3: What is the degree of Reach, Efficacy, Adoption, Implementation, 
and Maintenance of the eCPP intervention in pediatric primary care 
 
Study Design. The study will be a person-randomized clinical trial comparing the effects of the eCPP tablet-
based intervention to an enhanced usual care control condition (Table 1). Parents (n = 312) of 2- to 5-year-old 
children will be recruited from 4 primary care practices and randomly assigned to either the eCPP or control 
condition. Randomization will occur within sites at the participant level. Intervention parents (n = 156) will 
complete the 6 modules of the eCPP over a period of 3 months (~2 weeks/module; intervention phase) with 
ongoing access to the program (reference phase) throughout the remainder of the study period. Control 
condition parents (n = 156) will receive tablet-based parenting materials that they would normally receive at 
their primary care practice (e.g., information regarding nutrition, immunization, and exercise) and have access 
to the materials over the same intervention time period. Data collection in both conditions will occur at baseline 
and at 3, 6, and 12 months post-baseline. A descriptive design will be used to evaluate implementation of the 
eCPP from the perspective of practice sites, clinicians, and parents. Guided by the RE-AIM framework, we will 
use surveys, electronic health records, data usage metrics, and post-implementation assessments to evaluate 
the implementation process over the study period.  
Measures. 
Aim 1: Randomized Control Trial. All measures are written at the 5th grade reading level and 
administered on an Android tablet as computer assisted, self-interviews (CASIs). An audio option (audio-CASI) 
is available if parents prefer to listen to the questions. All control and outcome data are obtained by parent self-
report and will be collected at each of the 4 data collection time points (baseline, 3, 6, and 12 months post 
baseline). Previous findings from RCTs of the group-based CPP have found significant findings across three 
sources of data: parent self-report, teacher report and parent-child observation.14,24 In these studies, the parent 
self-report findings have consistently agreed with other methods of measuring parent and child outcomes. For 
this reason, the source of parenting and child outcome data for this study will be parent self-report alone. 
Control variables. We will collect information on social determinants of health that may affect parenting 
and child behavior outcomes. The social determinants of health are: general demographics, income and 
economic hardship, and neighborhood and community characteristics.69 Demographics (e.g., age, 
race/ethnicity, household structure) will be collected using a 21-item demographic inventory. Income and 
economic hardship will be assessed with a 9-item income inventory and an economic hardship scale (7 
items).70 In our current study, Cronbach’s alpha for the economic hardship scale was .68. Neighborhood and 
community characteristics will be collected using 9 items from the National Survey of Children’s Health.71 
Questions include perceived neighborhood social support, condition, and safety.  
Outcome variables. Primary parenting outcome variables include parenting behavior, self-efficacy, and stress. 
The primary child outcomes are child behavior problems and prosocial behavior. All survey data are obtained 
by parent self-report and will be collected at each of the 4 data collection time points (baseline, 3, 6, and 12 
months post baseline). Parenting behavior will include parenting discipline strategies and positive parenting 

behavior. Parenting discipline strategies will be measured using the 40-item Parent Questionnaire (PQ). Three 
scales – warmth, follow-through on discipline, and corporal punishment – using a 5-point scale measure how 
parents discipline their children. Cronbach’s alpha for the three subscales are .84, .80, and .68, respectively.14 
The PQ is a valid measure of parenting behavior and discipline skills in African American and Latino 
families.14,72 Positive parenting behavior will be assessed using the 21-item Parenting Young Children 
(PARYC) measure using a 7-point scale. There are three subscales for the PARYC: supporting positive 
behavior, setting limits, and proactive parenting. Cronbach’s alpha for the three scales are .78, .79, and .85, 
respectively.73 Parenting self-efficacy will be evaluated using the 17-item Parenting Sense of Competence 
Scale (PSOC). The PSOC has two subscales using 6-point scales: satisfaction (person's liking of the parenting 
role) and efficacy (person's perceived competence in the parenting role). Cronbach’s alpha for the total scale, 
satisfaction, and efficacy subscales are .79, .80, and .80, respectively.74,75 The PSOC is correlated with other 
measures of family life and child behavior, and the satisfaction subscale is strongly correlated with measures of 
child behavior, parent well-being, and parenting style.74,76 Parenting stress will be assessed with the Parenting 
Stress Index-Short Form (PSI-SF). The PSI-SF is a 36-item survey with three scales using a 5-point scale: 
parental distress, parent-child dysfunction, difficult child.77 Cronbach’s alpha is .90 for the total scale, .79 for 
parental distress, .80 for parent-child dysfunction, and .78 for difficult child.78 The PSI-SF is a valid measure of 
parenting stress in multicultural samples and for parents from lower socioeconomic groups.79,80  
We will measure child behavior problems using the Eyberg Child Behavior Inventory (ECBI).81 The ECBI is 
a 36-item scale designed to measure the presence and intensity of problem behavior. Each item is measured 
on two scales: the Problem Scale (dichotomous) and Intensity Scale (7-point). The ECBI is a valid measure of 
child behavior problems, with established convergent validity across racial and ethnic populations and 
economically and linguistically diverse samples.81-83 Cronbach’s alpha reliabilities of the Intensity and Problem 
Scale scores are.92 and .99.24 In addition, the Strengths and Difficulties Questionnaire (SDQ) will provide a 
measure of child behavior problems and prosocial behavior. The 25-item SDQ has a five-factor structure 
(emotional, conduct, hyperactivity-inattention, peer, and prosocial) using a 3-point scale. Mean Cronbach’s 
alpha for the SDQ is .73.84 The SDQ has been validated in multiple cultures and socioeconomic levels.85 
  Aim 2: Cost effectiveness analysis. Data for the cost effectiveness analyses will be drawn from 
invoices, staff report, study records, EHR, and eCPP electronic time stamps (Table 4). Resources used for the 
Table 4. Description of Resource and Cost Measures 
 
 
Cost Category 
Source 
Unit of Measure 
Translation to Costs 
eCPP costs 
eCPP delivery costs  
Invoice 
Participant 
Average cost/participant 
Orientation delivery costs 
EHR modification 
Programmer report 
Minutes/site 
Programmer average hourly wage 
Clinician time receiving study 
orientation 
Study records 
# of clinicians 
Minutes/clinician 
Clinician average hourly wage  
Time to lead orientation  
Study records 
Minutes  
Study staff average hourly wage 
Clinician costs 
Time to introduce study* 
EHR 
Minutes/ participant 
Clinician average hourly wage 
Parent costs 
Time spent learning about the study*  
EHR 
Minutes/ participant 
Parent average hourly wage 
Time spent completing eCPP modules 
digital time logs 
Minutes/ participant 
Participant average hourly wage  
EHR = Electronic Health Record; *time for introducing and learning about the study includes all parents (non-participants and 
participants), averaged across the # of participants. 
intervention include the eCPP costs (e.g., tablet, tech support, and maintenance costs), modification of the 
EHR, orientation for clinicians, and time for the clinician to introduce the study to parents during well-child 
visits. Participant costs are primarily the parents’ opportunity costs, representing the value of their time spent in 
the intervention or control condition, respectively. Participant costs include time spent completing the eCPP. 
Resources used will be translated into costs by multiplying the number of units with the cost per unit for each 
cost category. 
 Aim 3: RE-AIM data collection. Process tracking forms will be created to record the evaluation data, if 
not already part of the RCT outcome data (Table 5). These data will be collected monthly by study staff at each 
site during the recruitment phase from the 4 primary care practice sites, clinicians, and participants. 

Table 5. Data and Source for RE-AIM Evaluation 
RE-AIM component  
Data and Source 
REACH: Proportion and 
characteristics of parents 
from the practices that were 
introduced to and enrolled in 
the study.  
# of monthly well-child visits for children aged 2-5 over course of study recruitment (source: EHR) 
# of parents who were introduced to the study during a well-child visit (source: EHR) 
# of parents who enrolled in the study (consented/randomized) 
# of parents who were not introduced to the study and reason (source: EHR) 
Characteristics of parents enrolled in the study (source: demographics [C.5.a], Technology Inventorya)  
EFFICACY: Evaluation of 
intervention efficacy.  
Efficacy of the eCPP intervention is assessed as part of the RCT arm of the study by changes in 
parenting and child behavior. [Specific Aim 1- RCT (Hypothesis 1a)] 
ADOPTION: Characteristics 
of clinicians and practices 
oriented and agreeing to 
deliver the intervention (e.g., 
introduce the study to 
eligible parents).  
Clinician characteristics: MD or NP, specialty, years in practice, and current practices of managing 
parenting and child behavior concerns (source: Primary Clinician Surveyb). 
Site characteristics: # of patients aged 2-5, average time of well-child visits (source: EHR).  
# of orientation sessions per practice site and proportion of clinicians that attend the orientation 
(source: attendance records).  
IMPLEMENTATION: 
Assessment of the delivery 
of the introduction of the 
study by clinicians and 
associated costs. 
Proportion of clinicians that introduce the study to parents of patients aged 2-5 at well-child visits 
(source: EHR). 
Tracking and monitoring any implementation adaptations by the clinicians (source: clinicians). 
NOTE: Evaluation of implementation-related cost will occur as part of data collection and analysis of 
Aim 2 [Specific Aim 2- cost-effectiveness analysis]. 
MAINTENANCE: Evaluation 
of the practice site 
maintenance of 
implementation procedures 
and participant level 
maintenance of behavior 
change. 
Evaluation of maintenance of procedures across the study period at the practice sites (source: 
recruitment rates, # of parents introduced to the study monthly). 
# of follow-up orientation sessions for clinicians. 
Assessment of implementation challenges and facilitators, and sustainability of activities in clinical 
practice (source: post-implementation interviews with clinicians). 
Maintenance of behavior change over time will be assessed as part of the RCT arm of the study. 
[Specific Aim 1- RCT (Hypothesis 1b)]. 
EHR = Electronic Health Record. aThe Technology Inventory includes 18 items for parents to respond to about their use and 
ownership of technology. bThe Primary Clinician Survey86 is a 25-item measure developed by Metzler and colleagues for their 2014 
NIH-funded grant “Parenting Help Online Study” to assess pediatric primary care clinicians’ current practices in helping parents with 
child behavior problems and parenting concerns. The measure will be used with permission by the authors; no reliability data has 
been reported. (Appendix B)  
 
Analytic plan. 
Aim 1. Test the direct effects of the 6-module eCPP on parenting outcomes (parenting behavior, self-
efficacy, and stress) and child outcomes (child problem and prosocial behavior) compared to an enhanced 
usual care control condition among low-income parents with young children seen in primary care. HYPOTHESIS 
1A: Relative to parents in the control condition, parents in the eCPP condition will report greater improvements 
in parenting and child outcomes at the 3-month assessment. HYPOTHESIS 1B: The improvements in parenting 
outcomes and child outcomes the eCPP condition relative to the control condition will be maintained through 
the 6- and12-month assessments. 
All analyses for Aim 1 will be performed on an intent-to-treat basis. As multiple dependent variables are 
being examined, a repeated-measures multivariate analysis of variance (assessment x intervention condition) 
will be conducted to ensure control of experiment-wise Type-I error, and to perform an initial assessment of 
intervention effects. We will then test intervention efficacy using multilevel growth models90-92 with three levels 
(assessments [Level 1] within parent [Level 2] within primary care practice site [Level 3]). Intervention condition 
will be dummy coded (eCPP vs. control) at Level 2, and covariates at site-level (e.g., clinic size) or parent-level 
(e.g., age, race/ethnicity) will be included at the appropriate level of analysis. By including baseline measures 
of the outcome variable being tested as a covariate at Level 2 (parent level) and setting the intercept at the 3-
month assessment, we will be able to test Hypothesis 1A (intervention effects between baseline and 3-month 
assessment) as the main effect of intervention condition (i.e., controlling for baseline level of the outcome 
being tested). The condition x time interaction will then serve to test differential change in outcome measures 
following the 3 months of active intervention, and planned contrasts will be estimated to test Hypothesis 1B 
(maintenance of intervention effects at 6 and 12 months).  

Aim 2. Compare the cost-effectiveness of the eCPP intervention relative to control for parenting and child 
outcomes. HYPOTHESIS 2: The eCPP is cost-effective relative to control for parenting and child outcomes. 
We will assess cost-effectiveness of eCPP from the societal perspective, including costs borne by the 
program, health care system, and parent. We will conduct additional analyses from the program, health care 
system, and parent perspectives separately. For the cost measurement, quantities of resources used and their 
associated prices will be collected for the eCPP (either prices paid or value of the clinician time) and participant 
(i.e., value of participant’s time to use the eCPP). Program and parent costs will be calculated for the 12-month 
study duration and summed to calculate total cost per parent. All costs will be valued in 2015 dollars.  
For the effectiveness measurement, effectiveness will be measured using the parenting and child 
outcomes. Cost-effectiveness will be evaluated by combining the mean total cost per parent with effectiveness 
(parenting and child behavior outcomes). We will calculate the incremental cost-effectiveness ratio (ICER) for 
eCPP compared to the control condition, such that 𝐼𝐶𝐸𝑅= (𝐶1 −𝐶0) (𝐸1 −𝐸0)
⁄
, where C is cost and E is 
effectiveness. Subscript 1 denotes eCPP and subscript 0 denotes the control condition; 95% confidence 
intervals for the ICERs will be calculated to evaluate the uncertainty in these results.93-95 We will conduct one-
way and multi-way sensitivity analyses for the key parameters to evaluate whether the ICERs are sensitive to 
plausible changes in their values. The sensitivity analysis is a check on the robustness and will determine the 
key parameters impacting the ICERs. We will also plot acceptability curves based on varying threshold 
(willingness to pay) values for adherence and change in parenting and child behavior outcomes.  
Aim 3. Quantify the levels of program implementation of eCPP in primary care using the RE-AIM 
framework. RESEARCH QUESTION 3: What is the degree of Reach, Efficacy, Adoption, Implementation, and 
Maintenance of the eCPP intervention in pediatric primary care?  
Descriptive statistics will be calculated to identify achievements across each level of RE-AIM for each 
primary care site. We will develop a demographic, income, and technology use profile for the populations 
reached (i.e., parents enrolled in the study). We will use contingency table analyses and independent t tests to 
compare the demographic and income profiles to the characteristics of the population of the primary care sites 
service catchment communities using data from the U.S. Census. This analysis will allow us to assess the 
extent to which we are either over- or under-representing our priority populations (low-income families) from 
the local catchment area in this study. Technology ownership and use will be compared to data from national 
data sets from the Pew Research Center's Internet and American Life Project.96  
To determine the effectiveness of the implementation strategy and inform dissemination of the eCPP, a 
cascaded probability model will be constructed.97,98 This will be developed based on the following proportions: 
(a) the likelihood of clinician adoption at the site [proportion 1 (pr 1)], (b) the likelihood of implementation (study 
introduction) by clinicians at the site [pr 2], (c) the likelihood of parent enrollment in the study [pr 3], and (d) the 
likelihood of maintenance of behavior change by the eCPP participants [pr 4]. Pr 4 is defined as the proportion 
of participants whose parenting and child behavior outcomes are the same or improved at the 12-month data 
collection compared to the 6-month data collection. The estimated likelihood of parents being introduced to the 
program will be the product of pr1 and pr2. Second, the compound likelihood of parent enrollment in the study 
will be the product of pr1, pr2, and pr3. Finally, we will estimate the compound likelihood of maintaining 
behavior change by examining the product of pr1, pr2, pr3, and pr4. This analysis will provide us with two 
important pieces of information: (1) the probable benefits of the program in terms of how likely it is for parents 
to achieve maintenance of behavior change and (2) inform us of the phase of the implementation process (as 
operationalized by pr1, pr2, pr3, and pr4) which is likely to interfere with the dissemination of the eCPP.  
We will conduct a content analysis of the post-implementation interviews with the clinicians. We will use a 
coding-based system of content obtained from the interviews using the notes taken during the interview using 
Atlas.ti software. Atlas.ti allows for the identification, organization, and systematizing of codes that can then be 
assessed in the context of the clinician interviews. Themes will be summarized by site and used to inform the 
development of processes and procedures to support the implementation and dissemination of the eCPP in 
pediatric primary care practice sites.  
 
 

Bibliography & References Cited 
1. Children’s Defense Fund. Child poverty in America 2013: State analysis. http://www.childrensdefense.org 
/zzz-child-research-data-publications/data/child-poverty-in-america-1.pdf. Updated 2014. Accessed January 
12, 2015. 
2. Bauman LJ, Silver EJ, Stein REK. Cumulative social disadvantage and child health. Pediatrics. 2006; 
117(4):1321-1328. 
3. Guyer B, Ma S, Grason H, et al. Early childhood health promotion and its life course health consequences. 
Academic Pediatrics. 2009;9(3):142-149. 
4. Foster EM, Connor T. Public costs of better mental health services for children and adolescents. Psychiatric 
Services. 2005;56(1):50-55. 
5. Pearl ES. Parent management training for reducing oppositional and aggressive behavior in preschoolers. 
Aggression and Violent Behavior. 2009;14(5):295-305. 
6. Sanders MR, Markie-Dadds C, Tully LA, Bor W. The Triple P-Positive Parenting Program: A comparison of 
enhanced, standard, and self-directed behavioral family intervention for parents of children with early onset 
conduct problems. J Consult Clin Psychol. 2000;68(4):624-640. 
7. Gross D, Garvey C, Julion W, Fogg L. Preventive parent training with low-income, ethnic minority families of 
preschoolers. Handbook of parent training: Helping parents prevent and solve problem behaviors. 2007;3:5-24. 
8. Barlow J, Smailagic N, Ferriter M, Bennett C, Jones H. Group-based parent-training programmes for 
improving emotional and behavioural adjustment in children from birth to three years old. Cochrane database 
of systematic reviews (Online). 2010; 3. 
9. Garvey C, Julion W, Fogg L, Kratovil A, Gross D. Measuring participation in a prevention trial with parents of 
young children. Res Nurs Health. 2006;29(3):212-222. 
10. Ingoldsby EM. Review of interventions to improve family engagement and retention in parent and child 
mental health programs. J Child Fam Stud. 2010;19(5):629-645. 
11. Perrin EC. Commentary: Collaboration in pediatric primary care: A pediatrician's view. J Pediatr Psychol. 
1999;24(5):453-458. 
12. Gross D, Grady J. Group-based parent training for preventing mental health disorders in children. Issues 
Ment Health Nurs. 2002;23(4):367-384. 
13. Breitenstein SM, Shane J, Julion W, Gross D. Developing the eCPP: Adapting an evidence-based parent 
training program for digital delivery in primary care settings. Worldviews on Evidence-Based Nursing. in press. 
14. Gross D, Garvey C, Julion W, Fogg L, Tucker S, Mokros H. Efficacy of the Chicago Parent Program with 
low-income African American and Latino parents of young children. Prevention Science. 2009;10(1):54-65.  
15. National Research Council and Institute of Medicine. Preventing mental, emotional, and behavioral 
disorders among young people: Progress and possibilities. National Academies Press; 2009. O'Connell ME, 
Boat TF, Warner KE, eds. 
16. US Dept of Health and Human Services Office of the Surgeon General. National prevention strategy: 
America's plan for better health and wellness. 2011. 
17. Keane SP, Calkins SD. Predicting kindergarten peer social status from toddler and preschool problem 
behavior. J Abnorm Child Psychol. 2004;32(4):409-423. 
18. Scaramella LV, Leve LD. Clarifying parent-child reciprocities during early childhood: The early childhood 
coercion model. Clin Child Fam Psychol Rev. 2004;7(2):89-107. 
19. Shaw DS, Lacourse E, Nagin DS. Developmental trajectories of conduct problems and hyperactivity from 
ages 2 to 10. Journal of Child Psychology & Psychiatry. 2005;46(9):931-942. 
20. Foster EM, Prinz RJ, Sanders MR, Shapiro CJ. The costs of a public health infrastructure for delivering 
parenting and family support. Children and Youth Services Review. 2008;30(5):493-501. 
21. Campbell SB, Spieker S, Burchinal M, Poe MD. NICHD Early Child Care Research Network. Trajectories of 
aggression from toddlerhood to age 9 predict academic and social functioning through age 12. Journal of Child 
Psychology and Psychiatry. 2006;47(8):791-800. 
22. Lundahl B, Risser HJ, Lovejoy MC. A meta-analysis of parent training: Moderators and follow-up effects. 
Clin Psychol Rev. 2006;26(1):86-104. 
23. Wyatt Kaminski J, Valle LA, Filene JH, Boyle CL. A meta-analytic review of components associated with 
parent training program effectiveness. J Abnorm Child Psychol. 2008;36(4):567-589. 
24. Breitenstein SM, Gross D, Fogg L, et al. The Chicago Parent Program: Comparing 1‐year outcomes for 
African American and Latino parents of young children. Res Nurs Health. 2012;35(5):475-489. 

25. Thornton S, Calam R. Predicting intention to attend and actual attendance at a universal parent-training 
programme: A comparison of social cognition models. Clinical Child Psychology and Psychiatry. 
2011;16(3):365-383. 
26. Scott S, O'Connor TG, Futh A, Matias C, Price J, Doolan M. Impact of a parenting program in a high-risk, 
multi-ethnic community: The PALS trial. J Child Psychol Psychiatry Allied Disciplines. 2010;51(12):1331-1341. 
27. Heinrichs N, Bertram H, Kuschel A, Hahlweg K. Parent recruitment and retention in a universal prevention 
program for child behavior and emotional problems: Barriers to research and program participation. Prevention 
Science. 2005;6(4):275-286. 
28. Nix RL, Bierman KL, McMahon RJ, The Conduct Problems Prevention Research Group. How attendance 
and quality of participation affect treatment response to parent management training. J Consult Clin Psychol. 
2009;77(3):429-438. 
29. Werba BE, Eyberg SM, Boggs SR, Algina J. Predicting outcome in Parent-Child Interaction Therapy: 
Success and attrition. Behav Modif. 2006;30(5):618-646. 
30. Reyno SM, McGrath PJ. Predictors of parent training efficacy for child externalizing behavior problems - A 
meta-analytic review. J Child Psychol Psychiatry Allied Disciplines. 2006;47(1):99-111. 
31. Dumas JE, Wahler RG. Predictors of treatment outcome in parent training: Mother insularity and 
socioeconomic disadvantage. Behavioral Assessment. 1983;5(4):301-313. 
32. August GJ, Lee SS, Bloomquist ML, Realmuto GM, Hektner JM. Dissemination of an evidence-based 
prevention innovation for aggressive children living in culturally diverse, urban neighborhoods: The Early 
Risers effectiveness study. Prevention Science. 2003;4(4):271-286. 
33. Heinrichs N. The effects of two different incentives on recruitment rates of families into a prevention 
program. Journal of Primary Prevention. 2006;27(4):345-365. 
34. Metzler CW, Sanders MR, Rusby JC, Crowley RN. Using consumer preference information to increase the 
reach and impact of media-based parenting interventions in a public health approach to parenting support. 
Behavior Therapy. 2012;43(2):257-270. 
35. Lavigne JV, LeBailly SA, Gouze KR, et al. Treating oppositional defiant disorder in primary care: A 
comparison of three models. J Pediatr Psychol. 2008;33(5):449-461. 
36. Gross D, Johnson T, Ridge A, et al. Cost-effectiveness of childcare discounts on parent participation in 
preventive parent training in low-income communities. Journal of Primary Prevention. 2011;32(5-6):283-298. 
37. Cooper S, Valleley RJ, Polaha J, Begeny J, Evans JH. Running out of time: Physician management of 
behavioral health concerns in rural pediatric primary care. Pediatrics. 2006;118(1):e132-e138. 
38. Yarnall KSH, Pollak KI, Østbye T, Krause KM, Michener JL. Primary care: Is there enough time for 
prevention? Am J Public Health. 2003;93(4):635-641. 
39. Olson LM, Inkelas M, Halfon N, Schuster MA, O'Connor KG, Mistry R. Overview of the content of health 
supervision for young children: Reports from parents and pediatricians. Pediatrics. 2004;113(6 II):1907-1916. 
40. Meadows T, Valleley R, Haack MK, Thorson R, Evans J. Physician "costs" in providing behavioral health in 
primary care. Clin Pediatr. 2011;50(5):447-455. 
41. Bernal P, Estroff DB, Aboudarham JF, Murphy M, Keller A, Jellinek MS. Psychosocial morbidity: The 
economic burden in a pediatric health maintenance organization sample. Archives of Pediatrics and 
Adolescent Medicine. 2000;154(3):261-266. 
42. Talmi A, Fazio E. Commentary: Promoting health and well-being in pediatric primary care settings: Using 
health and behavior codes at routine well-child visits. Journal of Pediatric Psychology. 2012. 
43. Harrison JN, Cluxton-Keller F, Gross D. Antipsychotic medication prescribing trends in children and 
adolescents. Journal of Pediatric Health Care. 2012;26(2):139-145. 
44. Egger H. A perilous disconnect: Antipsychotic drug use in very young children. J Am Acad Child Adolesc 
Psychiatry. 2010;49(1):3-6. 
45. Zito JM, Safer DJ, Valluri S, Gardner JF, Korelitz JJ, Mattison DR. Psychotherapeutic medication 
prevalence in medicaid-insured preschoolers. J Child Adolesc Psychopharmacol. 2007;17(2):195-203. 
46. Child Trends. Well-child visits. www.childtrendsdatabank.org/alphalist?q=node/85. Updated May 2010. 
47. McLearn KT, Strobino DM, Minkovitz CS, Marks E, Bishai D, Hou W. Narrowing the income gaps in 
preventive care for young children: Families in healthy steps. Journal of Urban Health. 2004;81(4):556-567. 
48. Hickson GB, Altemeier WA, O'Connor S. Concerns of mothers seeking care in private pediatric offices: 
Opportunities for expanding services. Pediatrics. 1983;72(5):619-624. 
49. Berkout OV, Gross AM. Externalizing behavior challenges within primary care settings. Aggression and 
Violent Behavior. 2013;18(5):491-495. 

50. Polaha J, Dalton WT, Allen S. The prevalence of emotional and behavior problems in pediatric primary 
care serving rural children. Journal of Pediatric Psychology. 2011;36(6):652-660. 
51. The Henry J Kaiser Family Foundation. Summary of new health reform law. 
http://www.kff.org/healthreform/upload/8061.pdf. Updated April 15, 2011.January 24, 2012. 
52. Halfon N, Inkelas M. Optimizing the health and development of children. J Am Med Assoc. 
2003;290(23):3136-3138. 
53. Breitenstein SM, Gross D, Christophersen R. Digital delivery methods of parenting training interventions: A 
systematic review. Worldviews on Evidence-Based Nursing. 2014;11(3):168-176. 
54. Sameroff AJ, Fiese BH. Models of development and developmental risk. In CH Zeanah, Jr. (Ed.). 
Handbook of infant mental health. New York: Guilford; 2000.  
55. Patterson GR. Coercive family interactions. Eugene, OR: Castalia; 1982. 
56. Estabrooks PA, Harden SM, Allen KC. RE-AIM guidelines. In: Encyclopedia of behavioral medicine. 
Springer; 2013:1627-1630. 
57. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: A systematic review of use over time. Am J 
Public Health. 2013;103(6):e38-e46. 
58. Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek CJ. What does it mean to "employ" 
the RE-AIM model? Eval Health Prof. 2013;36(1):44-66. 
59. Glasgow RE, Phillips SM, Sanchez MA. Implementation science approaches for integrating eHealth 
research into practice and policy. Int J Med Inf. 2014;83(7):e1-e11. 
60. Estabrook B, Zapka J, Lemon SC. Evaluating the implementation of a hospital work-site obesity prevention 
intervention: Applying the RE-AIM framework. Health Promot Pract. 2012;13(2):190-197. 
61. Baggett KM, Davis B, Feil EG, et al. Technologies for expanding the reach of evidence-based 
interventions: Preliminary results for promoting social-emotional development in early childhood. Topics in 
Early Childhood Special Education. 2010;29(4):226-238. 
62. Self-Brown S, Whitaker D, Berliner L, Kolko D. Disseminating child maltreatment interventions: Research 
on implementing evidence-based programs. Child Maltreat. 2012;17(1):5-10. 
63. Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: A 
systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, 
and mode of delivery on efficacy. Journal of Medical Internet Research. 2010;12(1). 
64. Breitenstein SM, Gross D. Web-based delivery of a preventive parent training intervention: A feasibility 
study. Journal of Child and Adolescent Psychiatric Nursing. 2013;26(2):149-157. 
65. Raudenbush SW, et al. Optimal design software for multi-level and longitudinal research (Version 
3.01) [Software]. 2011. Available from www.wtgrantfoundation.org.  
66. Pyke-Grimm KA, Kelly KP, Stewart JL, Meza J. Feasibility, acceptability, and usability of web-based data 
collection in parents of children with cancer. Oncol Nurs Forum. 2011;38(4):428-435. 
67. Bellg AJ, Borrelli B, Resnick B, et al. Enhancing treatment fidelity in health behavior change studies: Best 
practices and recommendations from the NIH behavior change consortium. Health Psychology. 
2004;23(5):443-451. 
68. Resnick B, Bellg AJ, Borrelli B, et al. Examples of implementation and evaluation of treatment fidelity in the 
BCC studies: Where we are and where we need to go. Annals of Behavioral Medicine. 
2005;29(SpecSuppl):46-54. 
69. Shim R, Koplan C, Langheim FJ, Manseau MW, Powers RA, Compton MT. The social determinants of 
mental health: An overview and call to action. Psychiatric annals. 2014;44(1):22-26. 
70. Takeuchi DT, Williams DR, Adair RK. Economic stress in the family and children's emotional and 
behavioral problems. Journal of Marriage and the Family. 1991:1031-1041. 
71. Blumberg SJ, Foster EB, Frasier AM, et al. Design and operation of the national survey of children's health, 
2007. Vital Health Stat  1. 2012;(55):1-149. 
72. McCabe KM, Clark R. Family protective factors among urban African American youth. J Clin Child Psychol. 
1999;28(2):137-150. 
73. McEachern AD, Dishion TJ, Weaver CM, Shaw DS, Wilson MN, Gardner F. Parenting young children 
(PARYC): Validation of a self-report parenting measure. J Child Fam Stud. 2012;21(3):498-511. 
74. Ohan JL, Leung DW, Johnston C. The parenting sense of competence scale: Evidence of a stable factor 
structure and validity. Canadian Journal of Behavioural Science. 2000;32(4):251-261. 
75. Johnston C, Mash EJ. A measure of parenting satisfaction and efficacy. J Clin Child Psychol. 
1989;18(2):167-175. 

76. Rogers H, Matthews J. The Parenting Sense of Competence Scale: Investigation of the factor structure, 
reliability, and validity for an Australian sample. Australian Psychologist. 2004;39(1):88-96. 
77. Abidin R. Parenting stress index. 3rd ed. Odessa, FL: Psychological Assessment Resources, Inc; 1995. 
78. Roggman LA, Moe ST, Hart AD, Forthun LF. Family leisure and social support: Relations with parenting 
stress and psychological well-being in head start parents. Early Childhood Research Quarterly. 1994;9(3):463-
480. 
79. Reitman D, Currier RO, Stickle TR. A critical evaluation of the Parenting Stress Index-Short Form (PSI-SF) 
in a head start population. Journal of Clinical Child & Adolescent Psychology. 2002;31(3):384-392. 
80. Haskett ME, Ahern LS, Ward CS, Allaire JC. Factor structure and validity of the Parenting Stress Index-
Short Form. Journal of Clinical Child & Adolescent Psychology. 2006;35(2):302-312. 
81. Eyberg SM, Pincus D. Eyberg Child Behavior Inventory and Sutter-Eyberg Student Behavior Inventory-
revised: Professional manual. Psychological Assessment Resources; 1999. 
82. Burns GL, Patterson DR. Normative data on the Eyberg Child Behavior Inventory and Sutter-Eyberg 
Student Behavior Inventory: Parent and teacher rating scales of disruptive behavior problems in children and 
adolescents. Child and Family Behavior Therapy. 2001;23(1):15-28. 
83. Gross D, Fogg L, Young M, et al. Reliability and validity of the Eyberg Child Behavior Inventory with 
African-American and Latino parents of young children. Res Nurs Health. 2007;30(2):213-233. 
84. Goodman R. Psychometric properties of the Strengths and Difficulties Questionnaire. Journal of the 
American Academy of Child & Adolescent Psychiatry. 2001;40(11):1337-1345. 
85. Achenbach TM, Becker A, Döpfner M, et al. Multicultural assessment of child and adolescent 
psychopathology with ASEBA and SDQ instruments: Research findings, applications, and future directions. 
Journal of Child Psychology and Psychiatry. 2008;49(3):251-275. 
86. Metzler CW, Sanders MR, Rivara F, Christakis D, Rusby JC. Parenting help online study: Pediatrician 
questionnaire. Oregon research institute. 2014. 
87. Medical Research Council. Consort: Transparent reporting of trials. http://www.consort-
statement.org/consort-statement/overview0/. Updated 2012. Accessed January 27, 2012. 
88. Rubin D. Multiple imputation for nonresponse in surveys. New York: Wiley; 1987. 
89. Tukey J. Exploratory data analysis. Reading, MA: Addison-Wesley; 1977. 
90. Singer JD. Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth 
models. Journal of educational and behavioral statistics. 1998;23(4):323-355. 
91. Hox JJ. Multilevel analysis: Techniques and applications. Psychology Press; 2002. 
92. Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. Vol 1. 
Thousand Oaks, CA: Sage; 2002. 
93. Briggs AH, Wonderling DE, Mooney CZ. Pulling cost‐effectiveness analysis up by its bootstraps: A non‐
parametric approach to confidence interval estimation. Health Econ. 1997;6(4):327-340. 
94. Polsky D, Glick HA, Willke R, Schulman K. Confidence intervals for cost-effectiveness ratios: A comparison 
of four methods. Health Econ. 1997;6(3):243-252. 
95. Tambour M, Zethraeus N. Bootstrap confidence intervals for cost‐effectiveness ratios: Some simulation 
results. Health Econ. 1998;7(2):143-147. 
96. Pew Research Center. Pew research internet project. http://www.pewinternet.org/datasets/pages/7/ Web 
site.  
97. Raiffa H, Schlaifer R. Applied statistical decision theory. Hoboken, NJ: Wiley; 2000. 
98. Savage LJ. The theory of statistical decision. Journal of the American Statistical association. 
1951;46(253):55-67.