AN EVALUATION OF THE CULTURAL 

CONTEXTUALIZATION OF THE INCREDIBLE YEARS 

PARENTING PROGRAM FOR AFRICAN AMERICAN AND 

HMONG CULTURAL GROUPS 

April 15, 2016 

Prepared by: 

Jane Fields, PhD

Claire Harty, BA 

Christopher D. Desjardins, PhD 

This material is based upon work supported by the Corporation for National and Community Service 

(CNCS) under Grant No. 12SIHMN001 and Generation Next and Greater Twin Cities United Way. 

Opinions or points of view expressed in this document are those of the authors and do not necessarily 

reflect the official position of, or a position that is endorsed by, the Corporation, Generation Next, Greater 

Twin Cities United Way, or the Social Innovation Fund. 
Center for Applied Research and Educational Improvement 

University of Minnesota 2

Research, Development and Engagement to Improve Education 

Acknowledgements 

Please note that some language and detail have been taken directly from the final Social Innovation Fund 

Evaluation Plan (SEP) that was approved in June 2014 and authored by Beverly Dretzke, Ph.D., Research 

Associate, Center for Applied Research and Educational Improvement (CAREI); Muneer KarcherRamos, Director, Saint Paul Promise Neighborhood; Judy Ohm, Director of Early Childhood Services, 

Amherst H. Wilder Foundation; and Richard M. Lee, Ph.D., University of Minnesota Professor. We 

would also like to acknowledge the time that staff members from the United Way, the Saint Paul Promise 

Neighborhood, and the Wilder Incredible Years program spent helping us to understand the work that 

occurred during the grant period. 

How to Cite this Report 

Fields, J., Harty, C., & Desjardins, C. (2016, April). An Evaluation of the Cultural Contextualization of 

the Incredible Years Parenting Program for African American and Hmong Cultural Groups. Center for 

Applied Research and Educational Improvement, College of Education and Human Development, 

University of Minnesota. 

Contact Information 

Jane Fields, Ph.D. 

Center for Applied Research and Educational Improvement 

College of Education and Human Development 

University of Minnesota 

1954 Buford Avenue, Suite 425 

Saint Paul, MN 55108 

Telephone: (612) 624-4061

Email: jfields@umn.edu
Center for Applied Research and Educational Improvement 

University of Minnesota 3

TABLE OF CONTENTS 

TABLE OF CONTENTS ........................................................................................................................... 3

LIST OF TABLES ...................................................................................................................................... 5

LIST OF FIGURES .................................................................................................................................... 6

EXECUTIVE SUMMARY ........................................................................................................................ 7

Impact Evaluation Findings. ..................................................................................................................................... 8

Implementation Evaluation Findings. ....................................................................................................................... 9

Conclusions and Recommendations. ........................................................................................................................ 9

INTRODUCTION ..................................................................................................................................... 11

PROBLEM DEFINITION..............................................................................................................................................11

PROGRAM DESCRIPTION ..........................................................................................................................................12

PROGRAM THEORY, LOGIC MODEL, AND OUTCOMES OF INTEREST .........................................................................13

METHODS ................................................................................................................................................ 14

PURPOSE...........................................................................................................................14

EVALUATION QUESTIONS ........................................................................................................................................14

Impact Evaluation Questions ..............................................................................................................................14

Implementation Evaluation Questions ................................................................................................................15

DATA COLLECTION ..................................................................................................................................................15

Instruments .........................................................................................................................................................16

PSI-SF instrument ................................................................................................................................................... 16

SDQ instrument. ..................................................................................................................................................... 16

Parent Satisfaction Survey. ..................................................................................................................................... 17

Interviews. .............................................................................................................................................................. 17

STUDY DESIGN....................................................................................................................17

Impact Evaluation Design ...................................................................................................................................17

Implementation Evaluation Design .....................................................................................................................18

RESULTS .................................................................................................................................................. 20

SUMMARY INFORMATION ........................................................................................................................................20

IMPACT EVALUATION FINDINGS ..............................................................................................................................21

Sample ................................................................................................................................................................. 21

Tests of Pre/Post Differences ..............................................................................................................................22

Changes in Classification Categories .................................................................................................................23

IMPLEMENTATION EVALUATION FINDINGS ..............................................................................................................24

Program Dosage .................................................................................................................................................24

Implementation Fidelity ......................................................................................................................................25

Parents’ Satisfaction with the Program............................................................................................................................. 27

Survey Summary for the African American Cultural Contextualization Cohort ......................................................... 27

Overall Program. .................................................................................................................................................... 27

Teaching Format ..................................................................................................................................................... 28

Specific Parenting Techniques ................................................................................................................................ 28

Evaluation of Parent Group Leaders ....................................................................................................................... 28

Parent Group. .......................................................................................................................................................... 28

Open-Ended Responses .......................................................................................................................................... 28

Survey Summary for the Hmong American Cultural Contextualization Cohort ......................................................... 29

Overall Program. .................................................................................................................................................... 29

Teaching Format. .................................................................................................................................................... 29

Specific Parenting Techniques. ............................................................................................................................... 29

Evaluation of Parent Group Leaders. ...................................................................................................................... 29

Parent Group. .......................................................................................................................................................... 29
Center for Applied Research and Educational Improvement 

University of Minnesota 4

Open-Ended Responses. ......................................................................................................................................... 30

Cultural Contextualization ..................................................................................................................................30

Use of INTERFACE ........................................................................................................................................................ 30

Program Delivery ............................................................................................................................................................. 31

Video Vignettes ................................................................................................................................................................ 32

Key Staff Members’ Perspectives........................................................................................................................32

CONCLUSIONS ....................................................................................................................................... 33

REFERENCES .......................................................................................................................................... 36

APPENDICES ........................................................................................................................................... 38

APPENDIX A: BACKGROUND ON THE DEVELOPMENT OF THE HMONG CULTURAL CONTEXTUALIZATION................38

APPENDIX B: WILDER’S INCREDIBLE YEARS PROGRAM DESCRIPTIONS ..................................................................39

APPENDIX C: INCREDIBLE YEARS PARENT PROGRAMS CULTURAL CONTEXTUALIZATION LOGIC MODEL ..............40

APPENDIX D: IMPACT EVALUATION FINDINGS FOR THE AFRICAN AMERICAN CULTURAL CONTEXTUALIZATION ...41

APPENDIX E: IMPACT EVALUATION FINDINGS FOR THE HMONG CULTURAL CONTEXTUALIZATION ........................51

APPENDIX F: SURVEY RESULTS FOR THE AFRICAN AMERICAN CULTURAL CONTEXTUALIZATION..........................61

APPENDIX G: SURVEY RESULTS FOR THE HMONG CULTURAL CONTEXTUALIZATION .............................................65
Center for Applied Research and Educational Improvement 

University of Minnesota 5

LIST OF TABLES 

Table 1. Impact Evaluation Questions and Information Needed ................................................................ 17

Table 2. Implementation Evaluation Questions and Information Needed .................................................. 19

Table 3. Number of Cases Included in Impact Analysis ............................................................................. 21

Table 4. Summary of Effect Sizes on PSI-SF and SDQ for African American and Hmong Cultural 

Contextualization Cohorts ............................................................................................................ 22

Table 5. Summary of Classification Changes: Number of Children Whose Classification Category 

Improved, Stayed the Same, or Worsened ................................................................................... 23

Table 6. Program Information for African American and Hmong Cultural Contextualization Groups ..... 25

Table F1. The Overall Program (African American) .................................................................................. 61

Table F2. Teaching Format (African American) ........................................................................................ 62

Table F3. Specific Parenting Techniques (African American) ................................................................... 62

Table F4. Evaluation of Parent Group Leaders (African American) .......................................................... 63

Table F5. Parent Group (African American) .............................................................................................. 63

Table F6. Parent Group (Continued; African American) ............................................................................ 63

Table F7. Your Opinion (African American) ............................................................................................. 64

Table G1. The Overall Program (Hmong) .................................................................................................. 65

Table G2. Teaching Format (Hmong) ......................................................................................................... 66

Table G3. Specific Parenting Techniques (Hmong) ................................................................................... 66

Table G4. Evaluation of Parent Group Leaders (Hmong) .......................................................................... 67

Table G5. Parent Group (Hmong) ............................................................................................................... 67

Table G6. Parent Group (Continued; Hmong) ............................................................................................ 67

Table G7. Your Opinion (Hmong) .............................................................................................................. 68
Center for Applied Research and Educational Improvement 

University of Minnesota 6

LIST OF FIGURES 

Figure D1. PSI-SF Total Stress (African American) .................................................................................. 41

Figure D2. PSI-SF Parental Distress (African American) .......................................................................... 42

Figure D3. PSI-SF Parental Child Dysfunctional Interaction (African American) .................................... 43

Figure D4. PSI-SF Difficult Child (African American) .............................................................................. 44

Figure D5. SDQ Total Difficulties Score (African American) ................................................................... 45

Figure D6. SDQ Emotional Problems Subscale (African American) ......................................................... 46

Figure D7. SDQ Conduct Problems Subscale (African American) ............................................................ 47

Figure D8. SDQ Hyperactivity Subscale (African American) .................................................................... 48

Figure D9. SDQ Peer Problems Subscale (African American) .................................................................. 49

Figure D10. SDQ Prosocial Subscale (African American) ......................................................................... 50

Figure E1. PSI-SF Total Stress (Hmong) .................................................................................................... 51

Figure E2. PSI-SF Parental Distress (Hmong) ............................................................................................ 52

Figure E3. PSI-SF Parental Child Dysfunctional Interaction (Hmong) ...................................................... 53

Figure E4. PSI-SF Difficult Child (Hmong) ............................................................................................... 54

Figure E5. SDQ Total Difficulties Score (Hmong) .................................................................................... 55

Figure E6. SDQ Emotional Problems Subscale (Hmong) .......................................................................... 56

Figure E7. SDQ Conduct Problems Subscale (Hmong) ............................................................................. 57

Figure E8. SDQ Hyperactivity Subscale (Hmong) ..................................................................................... 58

Figure E9. SDQ Peer Problems Subscale (Hmong) .................................................................................... 59

Figure E10. SDQ Prosocial Subscale (Hmong) .......................................................................................... 60
Center for Applied Research and Educational Improvement 

University of Minnesota 7

EXECUTIVE SUMMARY 

AN EVALUATION OF THE CULTURAL CONTEXTUALIZATION OF THE 

INCREDIBLE YEARS PARENTING PROGRAM FOR AFRICAN AMERICAN AND 

HMONG CULTURAL GROUPS 

This report is based upon work supported by the Social Innovation Fund (SIF), a key White House 

initiative and program of the Corporation for National and Community Service (CNCS). Greater Twin 

Cities United Way received funds from the SIF to serve as a grantmaking intermediary in Minneapolis 

and Saint Paul. As part of a competitive selection process, the Amherst H. Wilder Foundation (Wilder) 

was selected as the fiscal agent for the Saint Paul Promise Neighborhood’s (SPPN) subgrant. 

Through the SIF award, Wilder and SPPN intended to deliver culturally contextualized versions of the 

Incredible Years program to African American and Hmong parents and children in the SPPN. 

The Center for Applied Research and Educational Improvement (CAREI) was awarded the contract to 

evaluate several of the seven nonprofit subgrantee programs that received SIF funding from the United 

Way. This evaluation report provides information about the cultural contextualization process and the 

effectiveness of the culturally-contextualized Incredible Years parent programs, specifically for African 

American and Hmong parents and children. The subgrant period reported here ran from April 15, 2013, 

through December 31, 2015, which includes Year 1 (April 2013-April 2014), Year 2 (April 2014-April 

2015), and a Year 2 no-cost extension (April 2015-December 2015). 

SPPN is part of the U.S. Department of Education’s Promise Neighborhood network. The SPPN, located 

in the Frogtown and Summit-University neighborhoods of St. Paul, has large African American and 

Hmong communities; however, a study by Wilder found low participation in early learning settings on the 

part of African American and Hmong families. The Wilder and SPPN’s theory of change hypothesizes 

that “cultural engagement is a critical piece of the education puzzle” and states that “all students and 

families have access to educational opportunities that honor and fully integrate their cultural practices, 

values, communication preferences, and learning styles” (Saint Paul Promise Neighborhood, 2015). With 

the SIF subgrant, SPPN and Wilder proposed to develop an African American contextualization of the 

Incredible Years parenting program, to refine a Hmong contextualization of the program, and to measure 

the effectiveness of the contextualizations on parenting skills and children’s social-emotional 

development. Incredible Years is an evidence-based program that emphasizes a strong parent-child 

relationship and enhancement of parenting skills. Each participating parent(s) is asked to select a focal or 

“interactive” child for the program. The interactive child joins the parent at the end of each session to 

work on techniques that were taught to the parent. The program provides childcare for the interactive 

child and siblings during each session. 

Prior to applying for SIF funding, Wilder worked with a University of Minnesota professor to adapt the 

Incredible Years model to be culturally-specific and relevant for Hmong parents and children. As part of 

the SIF funding, SPPN and Wilder wanted to further test this cultural adaptation for Hmong families, as 

well as develop and study a cultural adaptation for African American parents and children. 

The evaluation study was designed to describe the implementation of the contextualized versions of the 

Incredible Years parenting program and to measure the effect of the cultural contextualization on 

outcomes of African Americans and Hmong Americans. Because the Hmong contextualization was early 

in the testing stage and the African American contextualization had not yet been developed at the start of 
Center for Applied Research and Educational Improvement 

University of Minnesota 8

the subgrant period, the evaluation team determined that it was appropriate and feasible to obtain a 

preliminary level of evidence for each of these contextualizations by Year 2. 

The impact evaluation questions were:

● Do parents who participate in the culturally contextualized versions of Incredible Years show a 

reduction in parental stress and/or an increase in effective parenting skills, as measured by the 

Parenting Stress Index-Short Form (PSI-SF; Abidin, 1995)?

● Do children participating in the culturally contextualized versions of Incredible Years show 

improvements in social-emotional behaviors, such as emotional problems, conduct problems, 

hyperactivity, peer problems, and prosocial behavior as measured by the Strengths and 

Difficulties Questionnaire (SDQ; Goodman, 1997)? 

The implementation evaluation questions were:

● Are parents and children receiving the designated amount of treatment programming?

● Is the Incredible Years Program being implemented as intended? 

● Are the culturally contextualized versions of the Incredible Years model incorporating all of the 

key elements of the standard Incredible Years model (i.e., is the program still being implemented 

with fidelity after contextualization)?

● How satisfied are program staff, parents, and other stakeholders with the process of culturally 

adapting the Incredible Years program? 

● What are perceived to be the greatest successes and challenges of the contextualization process?

Incredible Years staff collected all data, including family entry forms, assessment results, parent 

satisfaction surveys, and other pertinent data. The Incredible Years facilitators administered two 

instruments to the parents at the beginning of program participation and again at the end to measure 

pre/post changes in the participating parents’ stress and children’s social-emotional development: the 

Parenting Stress Index-Short Form (PSI-SF) and the Strengths and Difficulties Questionnaire (SDQ).

Parent participants also completed a survey at the end of the program to gauge their satisfaction with 

Incredible Years. This information was shared with CAREI. CAREI evaluators also conducted semistructured interviews with key staff members to obtain information about the cultural contextualization, 

fidelity of implementation, and general perceptions about what worked or not throughout the grant period. 

Information from meetings between evaluation staff and the grantees was also included here.

Impact Evaluation Findings. Analyses were restricted to children and parents who participated in a 

culturally contextualized Incredible Years program (African American or Hmong) and who had complete 

data. During the grant period (April 2013-December 2015), 25 Incredible Years parenting groups were 

offered. Of those, 14 were community groups (i.e., not strictly African American or Hmong cultural 

contextualization groups), 4 were African American cultural contextualization groups, and 7 were Hmong 

cultural contextualization groups. The community groups included parents from a variety of cultural 

groups and were not culturally contextualized. For this evaluation study, all analyses were restricted to 

children and parents who participated in a culturally contextualized Incredible Years program and who 

had complete data. For the African American cultural contextualization cohort (the combined information 

from the 4 African American cultural contextualization groups), 13 pre/post scores on the PSI-SF and 8 

pre/post scores on the SDQ were included in the impact evaluation analysis. For the Hmong 

contextualization cohort, 26 pre/post scores on the PSI-SF and 19 pre/post scores on the SDQ were 

included.

Inferential and descriptive statistics were used to assess changes in parents and children. For both the PSISF and SDQ, pre/post differences were examined. In addition, the percent of scores falling into different 
Center for Applied Research and Educational Improvement 

University of Minnesota 9

categories from pretest to posttest were examined. To determine whether there were significant changes 

or effects as a result of participating in the Incredible Years culturally contextualized parenting programs, 

pretest scores and posttest scores on the PSI-SF and the SDQ were compared. Paired t tests were 

computed for the Hmong cultural contextualization cohort, and no significant differences were found for 

either instrument on the total scale or any of the subscales. Thus, there is no evidence in these data to 

suggest that there was any effect on the Hmong cohort. Although t tests were not conducted for the 

African American cultural contextualization cohort because the sample size was too small, effect sizes 

were calculated. The effect sizes were large and in the desired direction in three areas – PSI-SF Total 

Stress score, the PSI-SF Difficulty Child subscale, and the SDQ Prosocial subscale – which suggests that 

further examination of the impact of the African American cultural contextualization is warranted.

The number of changes in classification categories on the PSI-SF and SDQ was also examined (normal or 

clinical for PSI-SF; abnormal, borderline, or normal for SDQ). For all scales, the majority of children and 

parents remained in the same classification category from pretest to posttest. This is not surprising, given 

that no significant differences between pretest and posttest were detected. For one scale (SDQ Prosocial 

subscale for the Hmong cultural contextualization cohort), the number of children who moved into the 

“worsened” category at posttest was larger than the number in the “improved” category. 

Implementation Evaluation Findings. For the impact evaluation, descriptive statistics and qualitative 

analysis were used to describe implementation and participants’ reactions to the program The following 

aspects of implementation were addressed: program dosage, implementation fidelity, information about 

the cultural contextualizations, and key staff members’ perceptions.

During the subgrant period, 4 African American cultural contextualization groups and 7 Hmong cultural 

contextualization groups were held. The majority of the groups met for 7 weeks. Thirty-six African 

American and 43 Hmong parents participated in the culturally contextualized groups for a total of 79 

parents. In the African American contextualized cohort, there were 34 interactive children and 55 siblings. 

In the Hmong contextualized cohort, there were 40 interactive children and 45 siblings. 

Information about fidelity to the Incredible Years national curriculum was obtained through interviews 

and meetings with Wilder staff members who were responsible for implementing the Incredible Years 

program during the grant period. The evaluator was unable to observe program sessions in person, 

because it was determined that the evaluator’s presence would be intrusive and would inhibit the parents 

from fully participating in the program. Interviewees explained that the cultural contextualizations were 

adopted in such a way that the key elements of the standard Incredible Years model were preserved. 

The primary cultural contextualization occurred during the first week of programming. Although 

Incredible Years was implemented as usual using the standard Incredible Years lesson plans and 

activities, facilitators allowed time to incorporate cultural conversations into each session. No 

contextualization manual or curriculum was developed and no fidelity checks of the cultural 

contextualization occurred. However, interviewees noted that fidelity to the Incredible Years curriculum 

and programming was maintained using a variety of methods (e.g., completion of weekly facilitator 

checklists, observations by program coordinator, and so forth). Parents also completed a satisfaction 

questionnaire at the end of programming; in general, most parents were satisfied with the program. 

Conclusions and Recommendations. Although the subgrantee struggled with recruiting participants, 

during the grant period, they facilitated 25 groups that served 557 people – 202 parents and 355 children.

This shows that they were able to deliver the programming to many parents and children. However, in 

order to increase participation numbers, the programming was offered to community groups rather than 
Center for Applied Research and Educational Improvement 

University of Minnesota 10

strictly African American or Hmong contextualization groups. While this resulted in a larger number of 

parents and children receiving the Incredible Years programming, it did not provide large sample sizes for 

testing the hypothesis that culturally contextualized versions of the Incredible Years curriculum would 

improve outcomes for parents and children in the two cultural groups targeted for this intervention.

Several limitations in the dataset hindered our ability to further examine the data. Recommendations for 

future data collection include:

 Track daily attendance data;

 Track all groups in which a parent participates (dates, age groups, interactive child for each 

group);

 Attach parent data to their interactive children’s data;

 Enter the date an assessment was administered rather than the date it was entered into the 

database; and

 Enter item level information as well as scores for each assessment to allow the sample to be 

checked against the normative sample for each instrument.

Evaluating implementation was also challenging because evaluators relied on information provided by 

key staff members through meetings, documents, and interviews. Recommendations around 

implementation include:

 Create a manual for how to culturally contextualize the Incredible Years classroom while 

maintaining fidelity to the national curriculum;

 Clearly demonstrate how training on cultural contextualization occurs during the three-day 

Incredible Years facilitator training; and

 Adapt the Incredible Years materials to incorporate aspects of the cultural contextualization:

o Add items to the observer checklists that include ways in which facilitators add in 

cultural contextualization to the daily lesson;

o Create a rubric for the program manager to use when observing Incredible Years 

classrooms that includes cultural contextualization pieces;

o Add items to the parent weekly surveys and to the parent satisfaction survey that 

expressly ask parents whether they felt that their culture had been taken into account or 

that the terminology was understandable.

The purpose of the subgrant was to test whether an evidence-based program that did not necessarily have 

explicit cultural aspects could be modified to include more culturally appropriate pieces (e.g., 

terminology, video vignettes, examples) and then show the same or better outcomes for the participants. 

This question fits nicely with the theory of change put forth by the SPPN, which posits that “...culture is 

the missing piece of the puzzle in the education process.” The purpose of this grant was to validate the 

theory; however, because of the way it unfolded, the number of people participating in the cultural 

contextualization groups was lower than hoped. Project staff felt that the majority of the subgrant period 

was then spent on “getting the numbers,” rather than focusing on further refining the Hmong cultural 

contextualization and developing the African American version. However, in the process, Incredible 

Years program staff members reported that they learned several lessons: (1) adapting an evidence-based 

program is time-consuming work; (2) it takes time to build trust and rapport in communities, especially 

when you want to offer something new and test it at the same time; (3) recruitment strategies need to also 

be culturally contextualized; and (4) it may be best to partner with organizations that already serve the 

cultural groups your service is targeting, but do not currently offer the programming you provide. The test 

will be how Wilder and SPPN use the lessons they learned from participating in this project to continue to 

infuse culture into their programming. 
Center for Applied Research and Educational Improvement 

University of Minnesota 

 

1 For more information, see: nationalservice.gov/Innovation. 

2

 For more information, see: http://www2.ed.gov/programs/promiseneighborhoods/index.html. 

 11 

AN EVALUATION OF THE CULTURAL 

CONTEXTUALIZATION OF THE INCREDIBLE YEARS 

PARENTING PROGRAM FOR AFRICAN AMERICAN AND 

HMONG CULTURAL GROUPS 

INTRODUCTION 

This report is based upon work supported by the Social Innovation Fund (SIF), a key White House 

initiative and program of the Corporation for National and Community Service (CNCS). The Social 

Innovation Fund combines public and private resources to grow the impact of innovative, communitybased solutions that have compelling evidence of improving the lives of people in low-income 

communities throughout the United States.1

Greater Twin Cities United Way received funds from the SIF to serve as a grantmaking intermediary in 

Minneapolis and Saint Paul. As part of a competitive selection process, the Amherst H. Wilder 

Foundation (Wilder) was selected as the fiscal agent for the Saint Paul Promise Neighborhood’s (SPPN) 

subgrant. The Center for Applied Research and Educational Improvement (CAREI) was awarded the 

contract to evaluate several of the seven nonprofit subgrantee programs that received SIF funding from 

the United Way. 

Through the SIF award, Wilder and SPPN intended to deliver culturally contextualized versions of the 

Incredible Years program to African American and Hmong parent groups in the SPPN. This evaluation 

report will provide information about the cultural contextualization process and report on the 

effectiveness of the culturally-contextualized Incredible Years parent programs, specifically for African 

American and Hmong parents and children. The subgrant period reported here ran from April 15, 2013 –

December 31, 2015, which includes Year 1 (April 2013-April 2014), Year 2 (April 2014-April 2015), and 

a Year 2 no-cost extension (April 2015-December 2015). 

This report will provide background on the issue that was addressed with the proposed program model –

the cultural contextualization of the Incredible Years parenting program for African American and 

Hmong parent groups. The program theory, logic model, and outcomes of interest are also described 

because they helped to shape the implementation and impact evaluation questions and study design. 

Results for the implementation and impact evaluation are described followed by a conclusions section that 

summarizes the findings and provides recommendations. 

Problem Definition 

The Saint Paul Promise Neighborhood (SPPN) is part of the U.S. Department of Education’s Promise 

Neighborhood network whose vision “...is that all children and youth growing up in Promise 

Neighborhoods have access to great schools and strong systems of family and community support that 

will prepare them to attain an excellent education and successfully transition to college and a career.”2

The SPPN, located in the Frogtown and Summit-University neighborhoods of St. Paul, is host to a mix of 

many cultures, with the largest proportion of the neighborhood being from the African American and 
Center for Applied Research and Educational Improvement 

University of Minnesota 12 

3

 For more information on the Saint Paul Promise Neighborhood (SPPN), see: https://www.wilder.org/CommunityLeadership/Saint-Paul-Promise-Neighborhood/Pages/default.aspx. 

4

 For more information on the Incredible Years parenting program, see: http://incredibleyears.com/programs/parent/. 

Additional information can be found at the What Works Clearinghouse: 

http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=590. 

5

 For information on the Incredible Years Attentive Parenting program, see: 

http://incredibleyears.com/programs/parent/attentive-curriculum/. 

6

 There is a strong link between young children's social-emotional competence and their chances of early school 

success (Raver, 2002). Social-emotional development is also deeply rooted in culturally regulated child-rearing 

practices (Rubin, 1998). Studies have demonstrated that social-emotional knowledge has a critical role in improving 

children's academic performance and life-long learning (Zins, Bloodworth, Weissberg, & Walberg, 2004). Children 

who are aware of their emotions and have good planning skills by the time they enter school also have a lower risk 

for problems of aggression and anxiety disorders (Greenberg, Kusch, & Mihalic, 1998). Special programs that 

 

Hmong communities.3 Within the SPPN, there are glaring racial disparities in early childhood education 

participation. The 2011 SPPN Community Assessment (Karcher-Ramos, MartinRogers, Mueller, & 

Mattessich, 2011) found that only 5% of Hmong and 53% of African American children age birth through 

five living in the SPPN participated in formal early childhood learning programs. The SPPN Early 

Childhood Solution Action Group realized that low participation in early learning settings on the part of 

Hmong families is related to unfamiliarity with, and lack of trust of, non-school based centers, coupled 

with the cultural child rearing practices exhibited in mainstream early learning centers that may not be of 

interest to many Hmong parents. Among African American families, major barriers to participation in 

early learning settings include cost and transportation, as well as other factors. The SPPN’s Social 

Innovation Fund theory of change hypothesizes that “cultural engagement is a critical piece of the 

education puzzle” and states that “all students and families have access to educational opportunities that 

honor and fully integrate their cultural practices, values, communication preferences, and learning styles” 

(Saint Paul Promise Neighborhood, 2015). The belief is that cultural engagement will, in part, increase 

participation in programs because the curriculum is culturally-specific and culturally-relevant. 

For these reasons, the SPPN and Wilder partnered to deliver culturally contextualized curricula of the 

Incredible Years parenting program for these two cultural communities.4 With the Social Innovation Fund 

subgrant, SPPN and Wilder proposed to develop an African American contextualization, to refine a 

Hmong contextualization, and to measure the effectiveness of the contextualizations on parenting skills 

and children’s social-emotional development. 

Program Description 

Incredible Years is an evidence-based program that promotes emotional and social competence in young 

children and helps prevent aggression and emotional problems. Incredible Years parenting programs are 

offered to parents of babies (0-12 months), toddlers (1-3 years), preschoolers (3-6 years), and school age 

children (6-12 years). Incredible Years emphasizes a strong parent-child relationship and enhancement of 

parenting skills. Groups typically meet for 12-20 weekly 2-hour sessions. “Group sessions focus on: 

strengthening parent-child interactions, nurturing relationships, reducing harsh discipline, and fostering 

parents’ ability to promote children’s social, emotional, and language development” (Retrieved from: 

http://incredibleyears.com/programs/parent/). In addition, Incredible Years offers the Attentive Parenting 

program, which is a “6-8 session group-based ‘universal’ parenting program” that promotes “children’s 

emotional regulation, social competence, problem solving, reading and school readiness.”5 Educating 

parents about child development and parent-child interactions such as praising positive behaviors and 

coaching children through appropriate verbal expression of emotions can have a beneficial impact on 

social-emotional competence and, ultimately, on school readiness (Gray, Heberle, & Carter, 2012).6
Center for Applied Research and Educational Improvement 

University of Minnesota 13 

Prior to applying for SIF funding, Wilder worked with the University of Minnesota to adapt the Incredible 

Years model to be culturally-specific and relevant for Hmong parents and children. Examples of the ways 

in which the curriculum was culturally contextualized for Hmong families include the use of a Hmongonly group format and changes to the symbols used in the curriculum (e.g., participants found the use of a 

house image more culturally meaningful than the image of a pyramid to represent approaches to building 

parent/child relationships). (Appendix A provides background information on the development of the 

Hmong cultural contextualization.) As part of the SIF funding, SPPN and Wilder wanted to further test 

this cultural adaptation for Hmong families, as well as replicate the cultural adaptation for African 

American parents and children. 

In the subgrant proposal, Wilder proposed to deliver culturally-contextualized curricula of Incredible 

Years designed for Hmong parents and caregivers of children in four different age groups (0-12 months, 

1-3 years, 2-4 years, and 5-6 years) and to partner with the University of Minnesota to offer newly 

culturally-contextualized curricula for African American parents and caregivers of children in the same 

four age groups. (Appendix B offers a descriptive overview of the four Incredible Years program types 

offered by Wilder: the Parents and Babies Program, the Parents and Toddlers Program, the Parents and 

Toddlers Attentive Parenting Program, and the Early School Age (Preschool) Attentive Parenting 

Program.) Each participating parent(s) is asked to select a focal or “interactive” child for the program. 

The interactive child joins the parent at the end of each session to work on techniques that were taught to 

the parent. The program provides childcare for the interactive child and siblings during each 2-hour 

session. Originally, Wilder intended to recruit parents and children from SPPN-sponsored sites and 

programs and from other early learning settings affiliated with the SPPN Early Learning Network; 

however, offerings were expanded outside this network through the subgrant period. In addition to the 

cultural contextualization of the programming, Wilder and SPPN indicated that they would offer 

transportation assistance, childcare, a shared family meal, and a weekly incentive to help families 

overcome barriers to participation and encourage their ongoing participation.

Program Theory, Logic Model, and Outcomes of Interest 

The logic model for the Incredible Years cultural contextualizations (see Appendix C) is a modification of 

the logic model for the traditional Incredible Years Parent Program developed by Carolyn WebsterStratton at the University of Washington. The model presents program components and goals, program 

modalities, targeted risk and protective factors, proximate (short-term) outcomes, and distal (long-term) 

outcomes. 

The components of the program are the Incredible Years BASIC parent program and developmentally 

specific content for three age groups: babies, toddlers, and preschool children. The goal of the program is 

to enhance positive parenting interactions, coaching, attachment with children, and proactive discipline.

The program modalities are the specific strategies, methods, and techniques used to accomplish the 

program goals. These modalities include goal setting and self-monitoring; video monitoring; role play and 

behavioral practice; group support, discussion, and problem solving; cognitive emotional-regulation 

training; take-home practice and reading; and family meal and child care. 

 

promote social-emotional learning reduce aggression and increase prosocial behavior (Schonert-Reichl, Smith, 

Zaidman-Zait, & Hertzman, 2012; Weissberg, Kumpfer, & Seligman, 2003). 
Center for Applied Research and Educational Improvement 

University of Minnesota 14 

The program targets risk factors and protective factors. The risk factors are related to the parent, child, 

and family. The risk factors include parent personal/interpersonal problems; ineffective parenting skills; 

low monitoring; numerous stressors (e.g., challenging life events); low parent involvement with schools 

and teachers; and child biological factors (e.g., developmental delays). The program also targets 

protective factors related to the parents. These protective factors include positive, nurturing, 

developmentally appropriate parenting skills; parent support for the child’s social, emotional, and 

academic development; parent support networks; parent partnerships with teachers and schools; and 

effective communication skills, anger management, and problem solving. 

The short-term outcomes are targeted outcomes that the program has been shown in research to impact 

immediately and 1 to 3 years following program completion. Short-term outcomes are targeted in three 

areas: improved parenting and interpersonal skills, improved child behavior at home, and improved child 

behavior at school. The long-term outcomes for youth, designed to impact at long-term follow up during 

adolescence, are related to decreased antisocial behavior, less depression, and less likelihood of dropping 

out of school.

Research on the impact of parenting education programs has shown that such programs can significantly 

improve participants’ knowledge of effective parenting strategies (Winter, Morawska, & Sanders, 2012); 

the quality of parent-child interactions; the occurrence of a child’s positive behaviors in non-school 

settings; and parental depression (e.g., Niccols, 2009). Similarly, an investigation of the effects of the 

Incredible Years parent training program on outcomes for children aged 3 to 7 years and their parents 

reported significant improvements in children’s problem behaviors and social competence as well as 

increases in parental well-being and competence (McGilloway, Ni Mhaille, Bywater, Furlong, Leckey, 

Kelly, et al., 2012). In addition, research focused on school-related outcomes has found that a parenting 

intervention emphasizing positive behavior support, monitoring and limit setting, and enhanced 

relationship quality carried out with parents of children 2 to 3 years of age resulted in higher academic 

performance at ages 5 and 7.5 years when compared to the performance of children whose parents did not 

participate in the intervention (Brenna, Shelleby, Shaw, Gardner, Dishion, & Wilson, 2013). 

METHODS 

Purpose 

The evaluation study was designed to measure the effect of two Incredible Years cultural 

contextualizations on outcomes of African American and Hmong parents and interactive children. In 

addition, the evaluation was designed to describe the implementation of the contextualized versions of the 

Incredible Years parenting program.

Evaluation Questions 

IMPACT EVALUATION QUESTIONS

Prior to receiving SIF funding, the Hmong culturally contextualized program had been developed and 

offered once. Wilder also proposed, as part of this grant, to develop an African American cultural 

modification of the Incredible Years model. Because the Hmong contextualization was early in the testing 

stage and the African American contextualization had not yet been developed at the start of the grant 

period, the evaluation team determined that it was appropriate and feasible to obtain a preliminary level of 
Center for Applied Research and Educational Improvement 

University of Minnesota 15 

evidence for each of these contextualizations by Year 2, using the data from Years 1 and 2. For the impact 

evaluation, confirmatory and exploratory questions were proposed.7

The confirmatory questions for Years 1 and 2 are:

● Do parents who participate in the culturally contextualized versions of Incredible Years show a 

reduction in parental stress, as measured by the Parenting Stress Index-Short Form (PSI-SF; 

Abidin, 1995)?

● Do parents who participate in the culturally contextualized versions of Incredible Years exhibit an 

increase in effective parenting skills, as measured by the PSI-SF? 

● Do children participating in the culturally contextualized versions of Incredible Years show 

improvements in social-emotional behaviors, such as emotional problems, conduct problems, 

hyperactivity-inattention, peer problems, and pro-social behavior as measured by the Strengths 

and Difficulties Questionnaire (SDQ; Goodman, 1997)8

? 

IMPLEMENTATION EVALUATION QUESTIONS 

In this early stage of developing cultural contextualizations for the African American and Hmong 

communities of the standard Incredible Years model, the evaluation team also proposed studying program 

implementation. The implementation evaluation was designed to examine whether the project was 

implemented as proposed. The following questions were addressed by the implementation evaluation: 

● Are parents and children receiving the designated amount of treatment programming?

● Is the Incredible Years Program being implemented as intended? 

● Are the culturally contextualized versions of the Incredible Years model incorporating all of the 

key elements of the standard Incredible Years model (i.e., is the program still being implemented 

with fidelity after contextualization)?

● How satisfied are program staff, parents, and other stakeholders with the process of culturally 

adapting the Incredible Years program? 

● What are perceived to be the greatest successes and challenges of the contextualization process?

Data Collection 

Incredible Years staff collected all data, including family entry forms, assessment results, parent 

satisfaction surveys, and other pertinent data. Most of that information was then entered by Incredible 

 

7 Two exploratory questions were posed in the approved SIF evaluation plan, but they will not be addressed here. 

The first question, “Is the abbreviated series (6 or 7 weeks) of the culturally contextualized version of Incredible 

Years as effective as the longer series (10-14 weeks) as indicated by results on the outcome measures utilized to 

assess program impact?” could not be addressed because the sample sizes were not large enough to study dosage 

effects. The second question, “Are the instruments that have been utilized to assess outcomes of the traditional 

Incredible Years program (PSI-SF, SDQ, ASQ, and ASQ:SE) valid for assessing outcomes of the Hmong and 

African American culturally contextualized versions of Incredible Years?” could not be addressed because 

Incredible Years’ childcare workers did not complete the assessments or comparison of ratings as described in the

approved SIF evaluation plan; thus, information was not obtained that would allow for the planned instrument 

validation study.

8

In the final approved evaluation plan, it was also listed that the Ages and Stages Questionnaire (ASQ; Squires, 

Twombly, Bricker & Potter, 2009) and the Ages and Stages Questionnaire: Social-Emotional (ASQ:SE; Squires, 

Bricker, & Twombly, 2002) would be administered for younger children; however, only a small number were 

administered and there were administrative errors during administration so information from the ASQ and ASQ:SE 

was not available. 
Center for Applied Research and Educational Improvement 

University of Minnesota 16 

Years staff into CareLogic, the data system used by Wilder’s Child Development Center, and shared with 

the SPPN. Parents of Incredible Years participants signed a consent form that allowed Wilder to enter and 

store information in the SPPN database and gave permission for SPPN to share data with CAREI. 

The Incredible Years facilitators were asked to administer two instruments to the parents at the beginning 

of program participation and again at the end to measure pre/post changes in the participating parents’

stress and childrens’ social-emotional development. These assessments include: the Parenting Stress 

Index-Short Form (PSI-SF; Abidin, 1995) and the Strengths and Difficulties Questionnaire (SDQ; 

Goodman, 1997). The PSI-SF is completed by parents and measures the level of parental distress, positive 

parent-child interactions, and parent’s perceptions of the child’s behavior. The SDQ is also completed by 

parents and focuses on the child they have selected for class interaction time (the “interactive” child). The 

SDQ provides a measure of the child related to emotional problems, conduct problems, hyperactivity, 

peer relationship problems, and pro-social behavior. In addition to completing the pre/post parental stress 

and social-emotional development instruments, parent participants completed a survey at the end of the 

program to gauge their satisfaction with Incredible Years. 

INSTRUMENTS

PSI-SF instrument. The PSI-SF has three subscales (Parental Distress, Parent-Child Dysfunctional 

Interaction, and Difficult Child) consisting of 12 items each. Parents respond using a 5-point scale 

(strongly agree, agree, not sure, disagree, strongly disagree) to indicate how well each item represents 

their opinions.9 The Parental Distress subscale score indicates degree of distress arising from personal 

factors such as depression and stressful demands related to life role restrictions (e.g., “I feel trapped by 

my responsibilities as a parent”). The Parent-Child Dysfunctional Interaction subscale yields a score that 

indicates the level of dissatisfaction with the parents’ exchanges with their child and the extent to which 

parents feel their child does not meet expectations (e.g., “When I do things for my child, I get the feeling 

that my efforts are not appreciated very much”). The Difficult Child subscale provides an indication of 

parents’ perceptions of their child’s self-regulatory abilities (e.g., “My child reacts very strongly when 

something happens that my child doesn’t like”). In addition, a Total Stress score indicates the overall 

stress level experienced by the parents. 

SDQ instrument. The SDQ consists of 25 items that are divided among five scales of five items each. 

Subscale scores are generated for the five subscales: Emotional Problems, Conduct Problems, 

Hyperactivity, Peer Problems, and Prosocial Behavior. Example statements for the Emotional Problems 

subscale include, “Often complains of headaches, stomach-aches or sickness” and “Many worries or 

often seems worried.” Example statements for the Conduct Problems subscale include, “Often fights with 

other children or bullies them” and “Generally well behaved, usually does what adults request.”

Example statements for the Hyperactivity subscale include, “Restless, overactive, cannot stay still for 

long” and “Good attention span, sees chores or homework through to the end.” Example statements for 

the Peer Problems subscale include, “Picked on or bullied by other children,” and “Has at least one good 

friend.” Example statements for the Prosocial Behavior subscale include, “Considerate of other people’s 

feelings” and “Kind to younger children.” A Total Difficulties score is also provided. Respondents rate 

the extent to which statements reflect their child’s behavior on a 3-point Likert-type scale (not true, 

somewhat true, certainly true).10

 

9

 Parents are also measured on “Defensive Responding,” which is a validity subscale that indicates whether a parent 

is answering in a way that would make him/her look more favorable. 

10 For more information, see www.sdqinfo.org. 
Center for Applied Research and Educational Improvement 

University of Minnesota 17 

Parent Satisfaction Survey.11 Parents were administered the Incredible Years Parent Program Satisfaction 

Questionnaire (Attentive Parenting Program version) at the end of programming. This measure of parent 

satisfaction provides information for the implementation evaluation. Participating parents were asked 

questions about the overall program (7 questions), the teaching format (7 questions), the specific 

parenting techniques (7 questions), the parent group leader(s) (4 questions), and the parent group (4 

questions). Each question had one of several 7-point response scales; for example, parents were asked to 

rate the following statement, “My child’s social and emotional skills are” using a scale of considerably 

worse, worse, slightly worse, the same, slightly improved, improved, or greatly improved. The survey 

also includes three open-ended items about recommendations for improvement, need for further parenting 

assistance, and benefits of the program. 

Interviews. Semi-structured interview protocols were developed to ask program staff about the cultural 

contextualization, fidelity of implementation, recruitment, and general perceptions about what worked or 

not throughout the grant period. The CAREI evaluator conducted interviews with the Director of the Saint 

Paul Promise Neighborhood (SPPN); the Director of Wilder’s Child Development Center; the Incredible 

Years Coordinator; a Parent Education Recruiter and Community Outreach Coordinator and Incredible 

Years facilitator; two Incredible Years facilitators; and a University of Minnesota professor. In addition, 

several meetings were held with these key staff members, as well as two data specialists from Wilder, 

from June 2015 through December 2015. Information from these meetings was also included here. 

Study Design 

Because some parents and children left the program before all data were collected, missing data were 

present in the dataset. For this evaluation study, all analyses were restricted to children and parents who 

participated in a culturally contextualized Incredible Years program and who had complete data. 

Inferential and descriptive statistics were used to assess changes in social-emotional development, child 

behavior, parenting skills, and parental distress; descriptive statistics and qualitative analysis were used to 

describe implementation and participants’ reactions to the program. 

IMPACT EVALUATION DESIGN

For both the PSI-SF and SDQ, pre/post differences were examined. In addition, the percent of scores 

falling into different categories from pretest to posttest were examined (see Table 1 for more 

information). Only participants with PSI-SF pretest and posttest data were included in the PSI-SF 

analyses; only those with pre/post data on the SDQ were included in the SDQ analyses. 

Table 1. Impact Evaluation Questions and Information Needed 

Evaluation Question 

 Do parents who participate in the culturally 

contextualized versions of Incredible Years 

show a reduction in parental stress? 

 Do parents who participate in the culturally 

contextualized versions of Incredible Years 

Information Needed 

Parenting Stress Index-Short Form (PSI-SF)

 Pre/Post information completed by parents 

tested for significant differences between 

pretest and posttest on three subscale scores 

and Total Stress score

 

11 The final approved SIF evaluation plan indicated that weekly parent surveys would be administered and analyzed. 

The weekly survey was used for internal purposes only to monitor implementation fidelity. The information 

analyzed here is from the Incredible Years Parent Program Satisfaction Questionnaire (Attentive Parenting Program 

version) administered to parents at the end of the program. 
Center for Applied Research and Educational Improvement 

University of Minnesota 18 

Evaluation Question

exhibit an increase in effective parenting 

skills?

 Do children participating in the culturally 

contextualized versions of Incredible Years 

show improvements in social-emotional 

behaviors?

o SDQ. Will children participating in the 

Incredible Years program who are 

assessed by the SDQ show improvement 

in emotional symptoms, conduct 

problems, hyperactivity, peer problems, 

and prosocial behavior?

Information Needed

 Number in normal and clinical categories at 

pretest and posttest

 Number of pretest to posttest classification 

category changes

Strengths and Difficulties Questionnaire (SDQ)

 Pre/Post information completed by parents 

tested for significant differences between 

pretest and posttest on five subscale scores and 

Total Difficulties score

 Number in normal, borderline, or abnormal 

categories at pretest and posttest

 Number of pretest to posttest classification 

category changes

For the African American cohort that received the culturally contextualized Incredible Years 

programming, sample sizes on both the PSI-SF and the SDQ were too small to perform a t test analysis to 

test whether the differences between pretest and posttest were statistically significant for the overall scale 

score and subscale scores. However, effect sizes were calculated and are reported here. An appropriate 

effect size measure for the difference between two means is Cohen’s d, which is the mean difference in 

standard deviation units. A conventional interpretation of d is:

 0.8 = large effect size (almost an entire standard deviation) 

 0.5 = moderate effect size 

 0.2 = small effect size. 

For the Hmong cohort that received the culturally contextualized Incredible Years programming, matched 

pairs t tests were performed to test whether the differences between pretest and posttest were statistically 

significant for the overall scale score and subscale scores on both the PSI-SF and the SDQ. To test the 

robustness of our findings against violations of normality, a series of signed rank tests were also 

performed, and no differences were found between the t test results and the signed rank results (i.e., there 

was 100% agreement between the tests). Therefore, the t test findings will be presented in the results 

section for the Hmong cohort, along with information on effect sizes for the overall scale score and 

subscale scores. 

For both the African American and Hmong cohorts, the number of scores falling into the categories of 

“normal” and “clinical” at each test administration of the PSI-SF as well as the number of pre/post 

category shifts were examined. Similarly for the SDQ, for each cohort (African American and Hmong), 

the number of scores falling into the “abnormal,” “borderline,” or “normal” categories at both the pretest 

and posttest is reported for the Total Difficulties score and each subscale. The number of pre/post 

category shifts is also presented. 

IMPLEMENTATION EVALUATION DESIGN 

The purpose of the implementation evaluation was to verify that the project was implemented as 

proposed. The following aspects of implementation are addressed: program dosage; implementation 

fidelity; information about the cultural contextualizations; and key staff members’ perceptions of what 
Center for Applied Research and Educational Improvement 

University of Minnesota 19 

worked well, challenges, and lessons learned during the grant period. Table 2 shows the information 

obtained and reported for each implementation evaluation question. 

Table 2. Implementation Evaluation Questions and Information Needed 

Evaluation Question

 Are parents and children 

receiving the designated 

amount of treatment 

programming?

 Is the Incredible Years 

Program being implemented 

as intended? 

 Are the culturally 

contextualized versions of the 

Incredible Years model 

incorporating all of the key 

elements of the standard 

Incredible Years model (i.e., 

is the program still being 

implemented with fidelity 

after contextualization)?

 How satisfied are program 

staff, parents, and other 

stakeholders with the process 

of culturally adapting the 

Incredible Years program? 

 What are perceived to be the 

greatest successes and 

challenges of the 

contextualization process?

Information Needed

Program dosage information:

 Number of parents, interactive children, and siblings in the African 

American and Hmong culturally contextualized cohorts 

 Age ranges for interactive children

 Number of hours of programming and childcare

 Number and race/ethnicity of facilitators

 Dates of programming

Information on program delivery:

 Number of instructors who are African American and Hmong

 Number of instructors who speak the Hmong language

o Documents provided by Wilder

Information about implementation fidelity:

 Interview data that provides information about how fidelity of 

implementation was monitored

o Interviews with key staff members, including the Director of 

SPPN; the Director of Wilder’s Child Development Center; the 

Incredible Years Coordinator; a Parent Education Recruiter and 

Community Outreach Coordinator and Incredible Years 

facilitator; two Incredible Years facilitators; and a University of 

Minnesota professor

Information about satisfaction with Incredible Years programming:

 Incredible Years Parent Program Satisfaction Questionnaire 

(Attentive Parenting Program) results

o Data collected by, and provided to CAREI, by Wilder

Information about cultural contextualization:

 Interview data that provides information about the cultural 

contextualization that occurred

o Interviews with key staff members

 Document analysis and meeting notes

Feedback from staff members regarding: 

 Perceptions of program implementation (what worked well, 

challenges, lessons learned) and perceptions of impact on families 

o Interviews with key staff members

o Meeting notes

Information regarding program dosage was obtained from interviews and meetings with key project staff; 

from documents, provided by project staff, listing group and facilitator information; and from information 

in the CareLogic dataset that Wilder provided to CAREI.12 Information about implementation fidelity and 

the cultural contextualizations was obtained through interviews and meetings with key project staff. 

Another measure of implementation fidelity included here is the degree of satisfaction with the program 

 

12 The following aspects of program dosage that were proposed in the approved SIF evaluation plan will not be 

reported here due to lack of available data: attendance rates of parents and children, session locations, session times. 
Center for Applied Research and Educational Improvement 

University of Minnesota 20 

by participants. This was measured by the parent satisfaction questionnaire completed by parents at the 

end of the program.13 Finally, interviews and meetings with key project staff also provided information 

about successes, challenges, and lessons learned by the staff throughout the grant period. 

For the parent satisfaction survey, the percent of respondents selecting each response option was 

calculated for each survey question. In addition, the information provided in the open-ended questions 

was summarized. The qualitative data from the interviews was analyzed to identify common issues and 

themes and to provide additional context and information about the cultural contextualization process and 

implementation, as well as strengths and challenges experienced during the grant period. 

RESULTS 

Summary Information 

Recall that Incredible Years is a program offered to parents to strengthen parenting skills and to help 

parents foster their children’s social-emotional skills. Each participating parent(s) is also asked to select a 

focal or “interactive” child for the program. The interactive child joins the parent at the end of each 

session to work on techniques that were taught to the parent. The program provides childcare for the 

interactive child and siblings during the session. 

During the subgrant period (April 2013-December 2015), 25 groups were offered. Of those, 14 were 

community groups14 (offered to a mix of people), 4 were African American cultural contextualization 

groups, and 7 were Hmong cultural contextualization groups. The number of sessions varied across the 25 

groups: 

 The majority of the groups met for 7 sessions (14 groups), 

 3 groups met for 6 sessions, 

 3 met for 8 sessions, 

 1 met for 9 sessions, 

 1 met for 10 sessions, and 

 3 met for 12 sessions. 

Each group met once weekly for a 2-hour session. Nineteen facilitators ran the various groups – 8 were 

African American, 6 were Caucasian, 4 were Hmong, and 1 was Korean. In general, African American 

and Hmong facilitators worked with the African American and Hmong contextualization groups, 

respectively.15 In all, 557 people were served – 202 parents and 355 children. Because the purpose of this 

evaluation was to study the cultural contextualizations of the Incredible Years parenting program 

specifically offered to African American or Hmong parents and children, findings from only the 

contextualization groups will be reported here. For analysis purposes, information from the four African 

American cultural contextualization groups were combined and are called the African American cohort; 

 

13 The following information about implementation and the cultural contextualizations, proposed in the final 

approved SIF evaluation plan, was not provided for analysis: lesson plans, teaching materials, and handouts 

specifically related to the cultural contextualization; videotape recordings of group sessions; minutes of instructional 

time in the Hmong language (which could not be determined because facilitators spoke using a mix of English and 

Hmong).

14 The community groups included parents from a variety of cultural groups and were not culturally contextualized.

15 For one of the Hmong cultural contextualization groups, a Hmong and Caucasian facilitator worked together. 
Center for Applied Research and Educational Improvement 

University of Minnesota 21 

information from the seven Hmong cultural contextualization groups were combined and are called the 

Hmong cohort.16

Impact Evaluation Findings 

SAMPLE

For the impact evaluation, the following inclusion criteria were applied to the dataset provided to CAREI 

by Wilder – the parent or child had to be in either an African American or Hmong group (if the group 

name included “community” then the participant was excluded) and only interactive children were 

included and not siblings (data were not collected on siblings). In addition, for the inferential statistics, a 

parent’s data was included only if there was both a pretest and posttest. For cases in which a child had 

more than one pretest or posttest score on the SDQ or a parent had more than one pretest or posttest score 

on the PSI-SF, the data were excluded. Given those criteria, a total of 79 parents and 74 interactive 

children were included in the final dataset (see Table 3 for breakdowns by cultural contextualization 

cohort). Of the 79 parents, 39 parents had pretest and posttest data that could be included in the analysis 

for the PSI-SF (13 were African American and 26 were Hmong) and 27 parents had pretest and posttest 

data that could be included in the SDQ analysis (8 were African American and 19 were Hmong).17

Table 3. Number of Cases Included in Impact Analysis 

Cultural 

Contextualization 

Cohort

African American

Hmong

Total

Number of 

Parents

36

43

79

Number Included 

in PSI-SF Impact 

Analysis

13

26

39

Number Included 

in SDQ Impact 

Analysis

8

19

27

Number of 

Interactive 

Children*

34

40

74

Number 

of 

Siblings**

55

45

100

* Interactive children participate in the parent-child interaction sessions. 

**Siblings are for reference only; they were not administered any assessments or included in the study. 

Appendices D and E provide the pretest and posttest means, t tests (Hmong cohort only), Cohen’s d,

18 and 

changes in classification categories for the Total Stress score as well as for the three subscales (Parental 

Distress, Parental Child Dysfunctional Interaction, and Difficult Child) on the PSI-SF. The same 

information is provided for the SDQ Total Difficulties Score and for the five SDQ subscales (Emotional, 

Conduct, Hyperactivity, Peer Problems, and Prosocial). For both the PSI-SF and the SDQ, lower scores 

are better than higher scores (so a lower posttest score is indicative of an improvement on the measure) 

for all of the scale and subscale scores except for the SDQ Prosocial subscale (where a higher posttest 

score indicates an improvement). Appendix D shows the results for the African American 

contextualization cohort (see Figures D1-D10), and Appendix E shows the results for the Hmong 

contextualization cohort (see Figures E1-E10). 

 

16 Cohort refers to the total set of groups that received the cultural contextualization (e.g., the African American 

cohort refers to the four African American culturally contextualized groups). Group refers to the offering of the 

Incredible Years parenting program that occurred with different groups of parents and children over the subgrant 

period (e.g., there were seven Hmong culturally contextualized groups); group is a subset of cohort. Finally, session

refers to one of several meeting times when a group met (ranging from 6 sessions to 12 sessions). 

17 Dosage ranged from 6 sessions to 12 sessions for the two cultural contextualization cohorts. Although examining 

dosage effects would be interesting, the sample sizes were too small to study the impact of dosage. 

18 Cohen’s d was calculated as the absolute value of the mean of the differences between the pretest and posttest 

divided by the pooled standard deviation of these differences. Thus, when interpreting Cohen’s d, it is important to 

examine the pretest and posttest means to determine whether or not the effect was in the desired direction. 
Center for Applied Research and Educational Improvement 

University of Minnesota 22 

TESTS OF PRE/POST DIFFERENCES 

The purpose of the impact evaluation was to determine whether there were significant changes or effects, 

as determined by scores on the PSI-SF and the SDQ, as a result of participating in the Incredible Years 

culturally contextualized parenting programs for African American or Hmong parents and children. To do 

this, pretest scores and posttest scores on the PSI-SF and the SDQ were compared. Because of small 

sample sizes, t tests could not be performed for the African American contextualization cohort. Matched 

pairs t tests were performed on the Hmong cohort, but there were no significant differences from pretest 

to posttest. Effect sizes were computed for both cohorts, and Table 4 shows the summary effect size 

findings. Note that two large effect sizes were found for the African American contextualization cohort, 

suggesting that there was a large difference from pretest to posttest on the PSI-SF Difficult Child subcale 

(Cohen’s d = 0.95) and on the SDQ Prosocial subscale (d = 1.02). On the PSI-SF Difficult Child subscale, 

the posttest mean (20.8; SD = 6.2) was lower than the pretest mean (27.0; SD = 9.1), which shows an 

improvement in this area (see Figure D4, Appendix D). For the SDQ Prosocial subscale, where a higher 

score suggests an improvement, the posttest mean (8.6; SD = 1.3) was higher than the pretest mean (7.0; 

SD = 2.1) (see Figure D10). The effect size of 0.74 on the PSI-SF Total Stress score for the African 

American contextualization cohort is also worth examining further. Again, the posttest mean (62.0; SD = 

16.4) was lower than the pretest mean (74.5; SD = 21.6), suggesting an improvement (see Figure D1). 

These effect size findings suggest that with more power (larger sample sizes), a significant difference (or 

effect) could be detected. 

Table 4. Summary of Effect Sizes on PSI-SF and SDQ for African American and Hmong Cultural 

Contextualization Cohorts 

Cohort and Instrument Scale 

/ Subscale

Effect Size: Cohen’s d

Small (.2 - <.5) Moderate (.5 - <.8) Large (.8 and greater)

African American PSI-SF

 Total Stress 0.74

 Parental Distress 0.56

 Parent-Child 

Dysfunctional Interaction 0.26

 Difficult Child 0.95

Hmong PSI-SF

 Total Stress 0.36

 Parental Distress 0.22

 Parent-Child 

Dysfunctional Interaction 0.42

 Difficult Child 0.29

African American SDQ

 Total Difficulties Score 0.39

 Emotional Problems 0.25

 Conduct Problems 0.41

 Hyperactivity 0.42

 Peer Problems 0.24

 Prosocial 1.02

Hmong SDQ

 Total Difficulties Score 0.57
Center for Applied Research and Educational Improvement 

University of Minnesota 23 

Cohort and Instrument Scale 

/ Subscale

Effect Size: Cohen’s d

Small (.2 - <.5) Moderate (.5 - <.8) Large (.8 and greater)

 Emotional Problems 0.39

 Conduct Problems 0.41

 Hyperactivity 0.27

 Peer Problems 0.34

 Prosocial 0.34*

* This effect represented a negative effect (see Figure E10 on SDQ Prosocial subscale for the Hmong cohort).

CHANGES IN CLASSIFICATION CATEGORIES

The number of changes in classification categories on the PSI-SF and SDQ was also examined. Table 5 

shows the number of parents and children whose classifications (normal or clinical for PSI-SF; abnormal, 

borderline, or normal for SDQ) improved, stayed the same, or worsened from the pretest to the posttest. 

For all scales, the majority of parents and children remained in the same classification category from 

pretest to posttest. This is not surprising, given that no significant differences between pretest and posttest 

were detected. There is only one instance in which the number of children who moved into the 

“worsened” category at posttest was larger than the number in the “improved” category (for the SDQ 

Prosocial subscale for the Hmong cultural contextualization cohort, 5 children had scores in the 

“worsened” category and 2 had scores in the “improved” category; see Figure E10, Appendix E). 

For the African American contextualization cohort, across all scales on the PSI-SF, 17% of the 

classification categories improved from pretest to posttest, 77% stayed the same, and 4% worsened. On 

the SDQ, 29% of the classification categories improved, 63% stayed the same, and 8% worsened. 

For the Hmong contextualization cohort, across all scales on the PSI-SF, 20% of the classification 

categories improved from pretest to posttest, 73% stayed the same, and 7% worsened. On the SDQ, 15% 

of the childrens’ classification categories improved, 74% stayed the same, and 11% worsened.

Table 5. Summary of Classification Changes: Number of Children Whose Classification Category 

Improved, Stayed the Same, or Worsened 

Cohort and Instrument Scale 

/ Subscale

Classification Change from Pretest to Posttest

Improved Stayed the Same Worsened

African American PSI-SF (n=13)

 Total Stress 2 9 1

 Parental Distress 3 10 0

 Parent-Child 

Dysfunctional Interaction 2 10 1

 Difficult Child 2 11 0

Hmong PSI-SF (n=26)

 Total Stress 6 18 2

 Parental Distress 5 18 3

 Parent-Child 

Dysfunctional Interaction 6 18 2

 Difficult Child 4 22 0

African American SDQ (n=8)

 Total Difficulties Score 2 5 1
Center for Applied Research and Educational Improvement 

University of Minnesota 24 

Cohort and Instrument Scale 

/ Subscale

Classification Change from Pretest to Posttest

Improved Stayed the Same Worsened

 Emotional Problems 2 6 0

 Conduct Problems 2 5 1

 Hyperactivity 2 6 0

 Peer Problems 3 3 2

 Prosocial 3 5 0

Hmong SDQ (n=19)

 Total Difficulties Score 3 14 2

 Emotional Problems 2 16 1

 Conduct Problems 4 13 2

 Hyperactivity 2 16 1

 Peer Problems 4 13 2

 Prosocial 2 12 5

Implementation Evaluation Findings 

The implementation evaluation was designed to provide information that the project was implemented as 

proposed. The following sections provide information about implementation: program dosage, 

implementation fidelity, cultural contextualizations, and key staff members’ perspectives. 

PROGRAM DOSAGE

Because the purpose of this evaluation was to study the cultural contextualizations of the Incredible Years 

parenting program specifically offered to parents and children from African American or Hmong cultural 

communities, program dosage information will be reported only for the contextualization groups (see 

Table 619). This information is important in order to assess whether or not parents and children received 

an appropriate amount of treatment programming. 

From April 2013-December 2015 (the grant period), 4 African American cultural contextualization 

groups and 7 Hmong cultural contextualization groups were held. The majority of the groups met for 7 

weeks/sessions (5 groups); 1 met for 6 weeks; 2 met for 8 weeks; 1 met for 9 weeks; and 2 met for 12 

weeks. Each weekly session was 2 hours long. Four different facilitators worked with the African 

American cultural contextualization groups, all of whom were African American. Five different 

facilitators worked with the Hmong groups – 4 were Hmong and 1 was Caucasian. Two of the Hmong 

facilitators worked with 3 groups each, 1 worked with 2 groups, and 1 worked with 1 group; the 

Caucasian facilitator worked with 1 Hmong group. Although Incredible Years does not require two 

facilitators, Wilder has found that to be a successful strategy, so six of the eleven groups had two 

facilitators. 

Thirty-six African American and 43 Hmong parents participated in the culturally contextualized groups 

for a total of 79 parents (see Table 3 above). In the African American contextualized cohort (which 

 

19 The information reported in Table 6 was provided to CAREI by Wilder in a document, rather than being taken 

from the CareLogic dataset. The dates of programming in Table 6 do not always correspond to the begin dates 

provided in the CareLogic dataset for the participants determined to be eligible for inclusion in the impact 

evaluation.
Center for Applied Research and Educational Improvement 

University of Minnesota 25 

consists of those who participated in one of the 4 African American culturally contextualized groups), 

there were 34 interactive children and 55 siblings. In the Hmong contextualized cohort, there were 40 

interactive children and 45 siblings. The ages of the interactive children ranged from 1 to 7, with the 

majority between the ages of 3 and 6. This is to be expected, because Wilder focused on providing 

services to parents and children in this age range. 

Table 6. Program Information for African American and Hmong Cultural Contextualization Groups 

Cultural 

Contextualization 

Group

Dates Group 

Met

Number of 

Weeks 

Group Met

Hours of 

Programming

/ Child Care20

Race /

Ethnicity of 

Facilitator 1

Race /

Ethnicity of 

Facilitator 2

African American September 24 –

November 26, 

2013

7 14 hours African 

American

African 

American

African American January 21 –

April 22, 2015 12 24 hours African American

African 

American

African American September 9 –

October 28, 2015 7 14 hours African American

African 

American

African American April 14 – June 2, 

2015 8 16 hours African American None listed

Hmong October 8 –

November 26, 

2013

7 14 hours Hmong Hmong

Hmong April 9 – May 28, 

2014 7 14 hours Hmong None listed

Hmong January 21 –

March 5, 2015 7 14 hours Hmong None listed

Hmong January 21 –

April 22, 2015 12 24 hours Hmong None listed

Hmong April 13 – June 8, 

2015 6 12 hours Hmong Caucasian

Hmong October 1 –

December 10, 

2015

9 18 hours Hmong Hmong

Hmong October 2 –

December 11, 

2015

8 16 hours Hmong None listed

IMPLEMENTATION FIDELITY 

An examination of implementation fidelity was important to ensure that the core components of the 

Incredible Years program were retained, even when offering a culturally contextualized version of the 

Incredible Years program. Information about fidelity to the Incredible Years national curriculum was 

obtained through interviews and at meetings with Wilder staff members who were responsible for 

implementing the Incredible Years program during this grant period. The evaluator was unable to observe 

program sessions in person, because it was determined that the evaluator’s presence would be intrusive 

 

20 Child care was offered for the length of each session; each session was typically two hours long.
Center for Applied Research and Educational Improvement 

University of Minnesota 26 

and would inhibit the parents from fully participating in the program. An alternative was proposed in the 

final approved SEP to videotape sessions, but no videotaping of class sessions was conducted. 

Interviewees explained that the cultural contextualizations were adopted in such a way that the key 

elements of the standard Incredible Years model were preserved. One facilitator noted, “I don't think we 

really changed the curriculum because we still want to stay true to the IY curriculum because ... it's 

considered evidence-based so we still want to stay true to the topics and curriculum, and I think it works 

well when you just allow time to incorporate the conversations of culture into the session.” 

The primary cultural contextualization occurred at the beginning of the program – in the first and/or 

second meeting of the group through the use of a cultural contextualization resource that helps facilitators 

establish rapport and relationships with parents called INTERFACE. INTERFACE stands for 

Incorporating Narratives To Engage and Retain Families through Action, Cultural Connection, and 

Empowerment, and it is currently under development by Dr. Richard M. Lee at the University of 

Minnesota. 

Although Incredible Years was implemented as usual using the standard Incredible Years lesson plans 

and activities, facilitators allowed time to incorporate cultural conversations into each session (see below 

for more information about the cultural contextualization process). It should also be noted that because 

Wilder was having difficulty recruiting families for longer sessions, a decision was made during the grant 

period to offer only the Attentive Parenting Program to parents with children aged 3 to 5 years, which 

reduced the program to 7 weeks.21 This meant that the INTERFACE session was reduced from 2 weeks to 

1 week before Incredible Years programming began. 

No fidelity checks of the cultural contextualization occurred. However, fidelity to the Incredible Years 

curriculum and programming was maintained using the following methods: 

 All facilitators received the standard three-day Incredible Years training.22 The training included: 

o Training on the Incredible Years collaborative model and the Incredible Years 

curriculum; 

o Training on how to be a facilitator, which includes how to role play and how to help the 

parents and the interactive children work on techniques learned during class; and 

o Training on INTERFACE, which included information on how to conduct the first 

session and how to incorporate cultural components and sensitivity throughout the 

program. 

 The program manager met with facilitators before the start of any new group to help them prepare 

(to make sure they were ready to cover the curriculum, do role plays, discuss vignettes, and so 

forth for each meeting). 

 The program manager observed during the third or fourth session of the group. 

 Facilitators completed a standardized fidelity checklist at the end of each session. 

 Parents completed a feedback survey at the end of each session. The program manager provided 

each facilitator a summary of the parent survey responses after each session. 

 

21 The Incredible Years Attentive Parenting Program is a six-week program (see also Appendix B and 

http://incredibleyears.com/programs/parent/attentive-curriculum/). Typically, an additional week was added so that 

the INTERFACE cultural contextualization piece could be offered during the first week’s session.

22 Two interviewees mentioned that to become a certified Incredible Years parent facilitator, the facilitator must lead 

a certain number of groups and submit videotapes of the groups. No information was provided as to whether all or 

only some facilitators had received this certification. 
Center for Applied Research and Educational Improvement 

University of Minnesota 27 

 The program manager reviewed the facilitator checklists and the weekly survey data to see if 

there were any discrepancies or concerns that needed to be discussed with facilitators. 

 The program manager stayed in contact with facilitators via email and phone throughout the 

program offering. 

The Incredible Years program suggests that parents call each other during the week to serve as “buddies” 

as they practice the techniques they are learning in the program; however, the facilitators said that it could 

be difficult to get the parents to call each other, so often the facilitator made the phone calls. Also, as 

mentioned above, Incredible Years does not require two facilitators, but Wilder has found that practice to 

be a successful strategy for maintaining fidelity. Plus, as one interviewee noted, with two facilitators, 

“you have different point of views. Sometimes, you’re able to connect with a parent and somebody else 

maybe is more connected to one parent. So they can have closer connections with the facilitator.”

Parents’ Satisfaction with the Program

One aspect of fidelity includes assessing participants’ satisfaction with the program. To measure this, 

parents were administered the Incredible Years Parent Program Satisfaction Questionnaire (Attentive 

Parenting Program) at the end of the programming (during the last session). The survey was not modified 

to include any items specifically related to the cultural contextualization components, so no information is 

available regarding participants’ responses to the cultural contextualizations. The questionnaire includes 

questions about satisfaction with the overall program, teaching format, specific parenting techniques, 

group leader(s), the parent group, as well as three open-ended questions. Example questions and response 

options are: 

● “I feel that the approach used to strengthen my child’s social and emotional behaviors in this 

program is” (response options: very inappropriate, inappropriate, slightly inappropriate, neutral, 

slightly appropriate, appropriate, greatly appropriate);

● “Demonstration of parenting skills through the use of video vignettes was” (response options: 

extremely useless, useless, slightly useless, neutral, somewhat useful, useful, extremely useful); 

and

● “I feel the leader’s teaching was” (response options: very poor, poor, slightly below average, 

average, slightly above average, high, superior).

Wilder provided CAREI with anonymous participant level data for each survey question. The percent of 

respondents selecting each response option was calculated for each survey question. In addition, the 

information provided in the open-ended questions was summarized. A summary of the survey findings is 

presented below for each cultural contextualization cohort. For more information, see Tables F1-F7 in 

Appendix F for the African American contextualization cohort and Tables G1-G7 in Appendix G for the 

Hmong cohort.23

Survey Summary for the African American Cultural Contextualization Cohort 

Survey data were available for three of the African American cultural contextualization groups. A total of 

12 participants completed the survey (although not every respondent answered every question). 

Overall Program. Most respondents reported that their child’s social and emotional, pre-academic, selfregulation, and problem solving skills had improved slightly to greatly (see Table F1, Appendix F). No 

 

23 The percent of respondents answering each survey question is provided in the tables. Percent totals in the tables 

may be above or below 100% due to rounding error. 
Center for Applied Research and Educational Improvement 

University of Minnesota 28

respondents indicated that their child’s skills had decreased. However, 25% to 33% of respondents said 

that their child’s skills remained at the same level as before they started the program. When asked to rate 

their overall feelings about their progress at using skills taught by the program, 42% of the respondents 

felt optimistic. Likewise, 42% of respondents indicated that the approach used by this program was 

greatly appropriate to strengthen their child’s social and emotional behaviors. The majority of respondents 

(75%) responded that they would recommend this program to a friend or relative. Most respondents 

(83%) said they felt slightly to very positive about achieving their program goal.

Teaching Format. Eighty-four percent of respondents reported that the content of the information 

presented was somewhat to extremely useful (see Table F2). One respondent responded negatively, rating 

the content as useless. Responses indicated that there were mixed opinions about the utility of different 

program activities (e.g., video vignettes, group discussion, practice of play and coaching skills at home, 

reading chapters of Incredible Years texts, weekly handouts, and practice during group sessions). Some 

respondents rated five of the six activities as slightly useless or neutral. Despite that, the majority of 

respondents indicated that these activities were somewhat to extremely useful. Use of practice or role play 

during group sessions was rated lowest (42% selected slightly useless or neutral), and group discussion of 

parenting skills and weekly handouts were rated highest (84% rated these as somewhat useful, useful, or 

extremely useful).

Specific Parenting Techniques. Respondents were asked to rate specific parenting techniques: attentive 

child-directed play, academic and persistence coaching, social coaching, emotional coaching, teaching 

children to problem solve through play, and helping the child to control his/her anger (see Table F3). The 

majority of respondents rated these techniques as useful or extremely useful. For each of the techniques, 

one respondent rated the specific technique as either slightly useless or neutral. When asked to rate this 

overall group of techniques, 83% of respondents rated them as useful or extremely useful.

Evaluation of Parent Group Leaders. In the Incredible Years Parent Program Satisfaction Questionnaire 

for the Attentive Parenting Program, respondents are asked to rate each group leader individually. 

Because the purpose of this survey analysis was not to evaluate individual group leaders, but rather to 

gain a better understanding of how respondents perceived the group leaders, the information for all group 

leaders was combined (for this reason, there are a larger number of responses for each question in this 

section; see Table F4). No respondents rated the group leader on the negative end of the spectrum of 

response options. In fact, half of the respondents rated the leader’s teaching and preparation as superior;

the other half rated the leader as average, slightly above average, or high. All respondents were satisfied 

or extremely satisfied with the leader’s interest in and concern about the respondent and his or her child, 

and all felt the leader was helpful or extremely helpful.

Parent Group. Most respondents indicated that the parent group was supportive and that other group 

members displayed interest in the respondent and his or her child (see Tables F5 and F6). However, 80% 

of respondents said that they would not like to keep meeting as a parent group (respondents could select 

yes or no). Responses to the question, “How likely is it that you will continue meeting with one or more 

parents in your group?” were mixed; responses ranged from the lowest (highly unlikely) to highest (very 

likely) indicators on the spectrum of ratings. However, the majority of respondents (63%) said they were 

somewhat to very likely to meet with one or more group members again.

Open-Ended Responses. Parents are also asked to respond to three open-ended questions (see Table F7 for 

a complete list of responses). Six people answered the question, “How could the program have been 

improved to help you more?” Two people said the program could have covered more topics, and two 

others said the program could have been longer. One person commented that the curriculum was 
Center for Applied Research and Educational Improvement 

University of Minnesota 29 

“aggressive and ridiculous,” but another person said “I don’t think the program can be improved. I 

already think highly of the program.” Seven people answered the question, “At this time, do you feel the 

need for further parenting assistance?” Five people answered affirmatively, and two people answered 

negatively. Seven people answered the question, “What do you see as the main benefit of the Attentive 

Parenting Program?” There were a variety of responses, including learning new parenting techniques, 

helping children learn specific skills, and learning other parent’s perspectives. 

Survey Summary for the Hmong American Cultural Contextualization Cohort 

Survey data were available for four Hmong cultural contextualization groups. A total of 20 participants 

completed the survey (although not every respondent answered every question). 

Overall Program. The majority of respondents reported that their child’s social and emotional, preacademic, self-regulation, and problem solving skills had improved slightly to greatly (see Table G1, 

Appendix G). However, one respondent said that their child’s social and emotional, self-regulation, and 

problem solving skills had become slightly worse. Sixteen percent to 21% of respondents said that their 

child’s skills in these areas remained at the same level as before they started the program. When asked to 

rate their overall feelings about their progress at using skills taught by the program, most respondents 

(79%) felt slightly to very optimistic. Likewise, all respondents said that the approach used by this 

program was appropriate to strengthen their child’s social and emotional behaviors. All respondents 

indicated that they would recommend this program to a friend or relative. In addition, all respondents said 

they felt slightly to very positive about achieving their program goal.

Teaching Format. None of the responses concerning the teaching format were negative (see Table G2). 

All respondents indicated that the content of the information presented was useful or extremely useful. 

Likewise, all respondents felt that demonstration of parenting skills through the use of video vignettes, 

group discussion of parenting skills, and practice of play and coaching skills at home was somewhat to 

extremely useful. Some respondents rated reading chapters from Incredible Years texts, weekly handouts, 

and use of practice or role play during group sessions as neutral. 

Specific Parenting Techniques. Respondents were asked to rate the following specific parenting 

techniques: attentive child-directed play, academic and persistence coaching, social coaching, emotional 

coaching, teaching children to problem solve through play, and helping the child to control his or her 

anger (see Table G3). The majority of respondents felt that these techniques were somewhat to extremely 

useful. When asked to rate this overall group of techniques, all respondents rated them as useful or 

extremely useful. 

Evaluation of Parent Group Leaders. Respondents to this survey are asked to rate each group leader 

individually; however, because the purpose of this survey analysis was not to evaluate individual group 

leaders, ratings for all group leaders were combined (for this reason, there are a larger number of 

responses for each question in this section; see Table G4). No respondents rated the teaching on the 

negative end of the spectrum of responses options. However, one respondent rated the leader’s 

preparation as slightly below average. Having said that, the majority of respondents rated the leader’s 

teaching and preparation as superior (50% and 57%, respectively). Ninety-seven percent of respondents 

were slightly to extremely satisfied with the leader’s interest and concern in the respondent and his or her 

child, and all respondents found the leader to be helpful or extremely helpful in general. 

Parent Group. All respondents indicated that the parent group was supportive and that other group 

members displayed interest in the respondent and his or her child (see Tables G5 and G6). However, 88% 
Center for Applied Research and Educational Improvement 

University of Minnesota 30 

of respondents said that they would not like to keep meeting as a parent group (respondents could select 

yes or no). Responses to the question, “How likely is it that you will continue meeting with one or more 

parents in your group” ranged from the lowest (highly unlikely) to highest (very likely) of the response 

options. However, the majority of respondents (67%) said they were somewhat to very likely to meet with 

one or more group members again. 

Open-Ended Responses. Parents are also asked to respond to three open-ended questions (see Table G7 

for a complete list of responses). Thirteen people answered the question, “How could the program have 

been improved to help you more?” Four people said that the program was helpful and did not need 

improvement. Two people expressed that the program should be longer or cover less content to give more 

time for parents to learn. Other singular responses included that the leader should be more prepared and 

allow more advance notice for events outside of class, the videos should be translated into Hmong, and 

that the program was focused too much on one-to-one parent and child interactions. Fourteen people 

answered the question, “At this time, do you feel the need for further parenting assistance?” Eleven 

people answered negatively. The remainder of respondents expressed that the program was helpful and 

they could use further opportunities for practice. Seventeen people answered the question, “What do you 

see as the main benefit of the Attentive Parenting Program?” There was a range of responses, including 

the usefulness of videos, assignments, and examples; bonding with children; knowing themselves as 

parents; and helping their children become better people. 

CULTURAL CONTEXTUALIZATION

According to staff members, the cultural contextualization of the Incredible Years program occurred 

several ways, which will be discussed below. The information for this section was obtained through the 

analysis of meeting minutes, documents, and transcriptions of interviews with key staff members. 

Use of INTERFACE 

The INTERFACE resource was adopted as the framework for helping facilitators incorporate culture into 

the programming by establishing relationships with the parents. INTERFACE was designed as a twosession module that would precede Incredible Years programming, but once Wilder moved to the sixweek Attentive Parenting Program, INTERFACE was reduced to one session. An internal document 

provides the following additional information about INTERFACE: 

INTERFACE involves the articulation of family narratives to identify parenting values and goals. 

Then, using motivational interviewing techniques, parents will clarify these values and goals... 

the group facilitator will present the IY curriculum in a manner that matches the values, 

regulatory focus (i.e., promotion or prevention) and target (i.e., parent or child) of the parents.... 

Role induction techniques will be used to ensure parents understand their roles and 

responsibilities throughout the course of the IY program. Finally, cultural adaptations will be 

made through contextualization and augmentation based upon the information garnered from the 

... engagement module. (Judy Ohm, personal communication and internal document, 1/21/16)

The INTERFACE curriculum and training manual are under development, but Wilder incorporated 

aspects of it into the three-day Incredible Years facilitator training (as mentioned above). Facilitators used 

INTERFACE during the first week of programming. During this time, deeper connections are made with 

the parents. Facilitators discuss parenting issues with participants – how they were raised, their current 

parenting styles, their motivations as parents, their hopes for their children, family and cultural values, 

and so forth. One person commented that “the INTERFACE exchange is rich.”
Center for Applied Research and Educational Improvement 

University of Minnesota 31 

These parent narratives discussed during the first session were designed to increase parent engagement 

and buy-in to the program. The information was also used as a reference point to continue to incorporate 

culture into each session. 

We only use one session, it’s called the engagement session, and we use parts of the 

INTERFACE in that session to have parents talk about their own narratives and how do they 

parent, how were they parented, kind of trying to get them into the groove of there’s no right or 

wrong parenting. We just try to help them open up a little bit about what parenting is for them. 

Then we introduce what Incredible Years is and what we can provide and how we could possibly 

fill in things that they would like to learn about. (Facilitator)

Facilitators took notes throughout a number of Incredible Years parent sessions during the grant period. 

The notes were summarized to gain a better understanding of the cultural contextualization that occurred 

during programming (Yang, 2015). The findings show that “contextualization is not generalizable for all 

groups regardless of background but that there are themes and styles of parenting that come up more than 

once in a variety of groups regardless of composition” (Yang, p. 2). Common themes included: respect 

for adults, social and emotional skills, culture, physical discipline, traditions. Yang concluded by saying 

that “the amount of themes can attest to the fact that every group is different and contextualization should 

be fluid along with the dynamic of the groups composition” (p. 7). 

Program Delivery 

Facilitators were matched with the cultural group when possible. Thus, African American facilitators 

were hired to facilitate the African American cultural contextualization groups; Hmong facilitators who 

speak Hmong were hired to facilitate the Hmong contextualization groups. 

Speaking in the Hmong language allowed facilitators an opportunity to translate key terminology in ways 

that would be more understandable. One of the Hmong facilitators explained that although Incredible 

Years is considered culturally competent, the curriculum cannot always be translated directly and still be 

understood by people from different cultural groups. For example, she said the concepts of pretend and 

imaginative play are difficult to translate in a way that would be understood by people in the Hmong 

culture. 

I think something I would say in terms of contextualization is that the national Incredible Years 

curriculum considers it as culturally competent across the board. But we’ve found that it’s not 

really, especially with the Hmong community, because for instance if I was to translate a lot of 

the vignettes, the concepts they talk about in those vignettes sometimes are not culturally 

competent. If I spoke about star wars and fighting out in space, I can’t translate that directly into 

Hmong and have it make sense.... So what I do is kind of modify concepts, not even modify, but 

try to relate it to what the parent understands or what the community understands. (Facilitator)

Thus, an important piece of the cultural contextualization was to use language and terminology that would 

be more meaningful and relevant to the cultural group. For example, a common metaphor used in the 

Incredible Years programming is the pyramid that depicts the parenting strategies that should be used 

most frequently at the bottom of the pyramid (e.g., play, praise and incentives, and effective limit setting) 

and the parenting techniques that should be used sparingly at the top of the pyramid (e.g., ignoring, 

distracting, redirecting, and time outs). The facilitators discussed how they used a house metaphor or a 

gardening metaphor instead for the cultural groups. They built in time to discuss how the pyramid model 
Center for Applied Research and Educational Improvement 

University of Minnesota 32 

may not be as relevant in their traditions and then provided another metaphor that might be more relevant 

but that still emphasized the Incredible Years model. 

Video Vignettes 

Video vignettes are used throughout the program. Facilitators discussed how the people depicted in the 

videos are not very diverse and the vignettes are in English. They felt that in order to do a cultural 

adaptation, the videos need to look more like the people in the group and be in the language of the culture. 

One facilitator said, “...let's get to the video, the all-white videos. You know I think if we're going to do a 

cultural course it would make sense that we had that cultural piece—you know that we see a Hmong 

family, we see a Hispanic family, we see a Somalian family, and we see an African American family. 

That would have been nice.” Thus, for the Hmong groups, facilitators at first would translate the clips as 

the video played. Because it was quite challenging to translate as the video played, a Wilder staff member 

translated the videos into Hmong and recorded the translations over the videos. The idea was that the 

recorded translation would play while the video played; however, the voice-over was not very successful. 

A facilitator stated, “...when you translate to Hmong, things are a little longer, so sometimes the video 

would end and then the voiceover continued because it’s not over yet. So it was a little confusing for the 

parents.” One facilitator believes it would be better to play the video in English or read the translation of 

the video in Hmong. Another said that it might be better to use more role play by the facilitators to depict 

the lessons from the videos. 

KEY STAFF MEMBERS’ PERSPECTIVES

Several key staff members acknowledged that, throughout the subgrant period, the initiative had trouble 

meeting their goals related to the number of African American and Hmong cultural contextualization 

groups held and the total number of parents and interactive children who participated. Some interviewees 

attributed this, in part, to “misperceptions around the type of grant it was.” One interviewee reported that 

the programming team thought, “It was an implementation grant in our viewpoint versus evaluation,”

which is a misconception about the SIF program. When the grant was awarded in April 2013, the Hmong 

contextualization had been offered once and the African American contextualization had not yet been 

developed; thus, Incredible Years contextualized programming was not offered until September 2013. 

One interviewee reported, “And so from April to September we were working on ‘we have to get people 

trained, we have to...’ You know there were a lot of things that had to happen in order to for us to be 

ready and so that’s why we didn’t actually pull groups until September.” Interviewees also talked about 

other implementation issues that resulted in them getting behind in their target numbers, such as a winter 

of bad weather and class cancellations, space issues, and the need to hire and train new facilitators. 

Although they acknowledged the difficulty they had in reaching their target numbers, the interviewees felt 

that several positive outcomes emerged from the grant funding. One person said: 

I think that we did miss our mark obviously, our numbers weren’t there. I think we have a good 

experience to talk about how that happened.... I mean I’m sure it was somewhat disappointing 

from the United Way’s part because we weren’t making our numbers, but yet, we were having an 

impact, we do have an impact ... look at the things we can validate. Can we validate that we’re 

working in cultural communities? Yes. Can we validate that the parents are coming in with a high 

level of stress when they’re living in poverty? ... And to know that that piece of it is yes, I mean 

you know the answer is yes, even though the numbers are small. (Interviewee)
Center for Applied Research and Educational Improvement 

University of Minnesota 33 

Although the impact evaluation findings did not show a significant change in parents or children, 

interviewees reported that they receive comments from parents about how they enjoyed the programming 

and felt that it was beneficial. Interviewees also described how they will see a parent who has gone 

through the program at Wilder or in the community and witness the parents saying or doing things that 

were modeled in the class. Another interviewee described how the people hired to be interpreters for the 

class also found it beneficial: “They would talk afterwards. They would say, ‘I’m going to use this at 

home.’ So [it was as if] they were going through the class.” Another interviewee said, “I feel like parents 

who go through it, they really say they like it and they benefited from that, so I think it's a great program 

for parents.”

Second, having the subgrant “frankly provided a testing ground for taking a program that wasn’t specific 

to Hmong or African American and gave SPPN (Saint Paul Promise Neighborhood) sort of a testing 

ground to figure out kind of how to do that.” This included changing the recruitment strategies throughout 

the subgrant period. One interviewee noted, “...It really is looking at reaching across the community and 

figuring out the pieces that are effective with various cultures, with various agencies, different parents...” 

Early on, Incredible Years administration staff members were trying to coordinate the programming 

(including the cultural contextualization components) and recruit parents. Recruiting has included posting 

flyers in the community and attending school and community resource fairs. Recruiting has also included 

building relationships with organizations that could refer parents to Incredible Years, such as Head Start 

and schools. During the grant period, they hired a Hmong parent navigator, whose job title quickly 

changed to “Recruiter and Community Outreach Coordinator.” She is responsible for developing 

marketing strategies to promote community interest. One interviewee stated, “culturally-based 

recruitment strategies matter.” She is promoting partnerships with other organizations with similar 

missions that do not provide parent education but would like that as part of their programming. One 

interviewee noted that the key lesson learned here is that it takes time to learn about community needs and 

to build relationships with community partners. She noted the importance of gaining trust and respect in 

the communities, so that you become a trusted resource for providing services. For example, if you are 

asking an organization to refer clients to your program, “it’s long-term—building relationships and trust 

because we’re really asking for them to hand over their clients, and they want to make sure that it’s 

quality, that we’re going to take care of them, we’re not going to alienate them.” 

CONCLUSIONS 

Although the grantees struggled with recruiting participants, during the grant period they facilitated 25 

groups that served 557 people – 202 parents and 355 children. This shows that as they honed their 

recruitment strategies, they were able to deliver the programming to more parents and children. However, 

in order to increase participation numbers, the programming was offered to community groups rather than 

strictly African American or Hmong contextualization groups. While this resulted in a larger number of 

parents and children receiving the Incredible Years programming, it did not provide larger sample sizes 

for testing the hypothesis that culturally contextualized versions of the Incredible Years curriculum would 

improve outcomes for parents and children in the two cultural groups targeted for this intervention. 

Again, because the purpose of this evaluation was to study the cultural contextualizations of the 

Incredible Years parenting program specifically offered to African American or Hmong cultural groups, 

findings from only the cultural contextualization cohorts were reported here. 

For the African American cultural contextualization cohort, 13 pre/post scores on the PSI-SF and 8 

pre/post scores on the SDQ were included in the impact evaluation analysis. For the Hmong 

contextualization cohort, 26 pre/post scores on the PSI-SF and 19 pre/post scores on the SDQ were 
Center for Applied Research and Educational Improvement 

University of Minnesota 34

included. Thus, there was not much power to detect a difference between pretest and posttest scores. 

Paired t tests were computed for the Hmong cultural contextualization cohort, and no significant 

differences were found for either instrument on the total scale or any of the subscales. Thus, there is no 

evidence to suggest that there was any effect on the Hmong cohort. Although t tests were not conducted 

for the African American cultural contextualization cohort, effect sizes were calculated. The effect sizes 

were large and in the desired direction in three areas, which suggests that further examination of the 

impact of the African American cultural contextualization is warranted. There was a large difference (in 

the desired direction) from pretest to posttest on the PSI-SF Total Stress score (Cohen’s d = 0.74), the 

PSI-SF Difficulty Child subcale (d = 0.95), and on the SDQ Prosocial subscale (d = 1.02). 

Several limitations in the dataset also hindered our ability to further examine the data. For example, there 

were several instances where a parent had more than one pretest, posttest, or both. Because it was often 

not clear in the dataset which pretest/posttest scores were administered first, no data for these parents 

were included in the analysis. There were also discrepancies in the data; for example, group begin/end 

dates in the dataset did not always match up with dates provided to CAREI by Wilder. No daily 

attendance information was provided, so even if the samples had been larger, there was no way to 

examine dosage effects. Finally, it would have been interesting to connect parents to children in the 

dataset in order to examine the connection between PSI-SF and SDQ scores or to connect parents’ 

satisfaction levels as measured by the survey with their scores on the PSI-SF. Recommendations for 

future data collection include:

 Track daily attendance data;

 Track all groups in which a parent participates (dates, age groups, interactive child for each 

group);

 Attach parent data to their interactive children’s data;

 Enter the date an assessment was administered rather than the date it was entered into the 

database; and

 Enter item level information as well as scores for each assessment to allow the sample to be 

checked against the normative sample for each instrument.

Evaluating implementation was also challenging because evaluators had to rely on information provided 

by key staff members through meetings, documents, and interviews. Evaluators did not observe (either inperson or through videotape) weekly sessions in order to examine implementation fidelity to the 

Incredible Years curriculum or to see how the cultural contextualization occurred. The contextualization 

pieces were not documented in the form of a manual or curriculum materials. The record of the 

contextualization is from interviews with key staff members and an analysis of notes taken during 

sessions. The lack of specific information about the cultural contextualization also makes it hard to 

attribute any effect, if one had been detected, to the standard Incredible Years program or to the 

Incredible Years plus contextualization condition. Recommendations around implementation include:

 Create a manual for how to culturally contextualize the Incredible Years classroom while 

maintaining fidelity to the national curriculum;

 Clearly demonstrate how training on cultural contextualization occurs during the three-day 

Incredible Years facilitator training; and

 Adapt the Incredible Years materials to incorporate aspects of the cultural contextualization:

o Add items to the observer checklists that include ways in which facilitators incorporate

cultural contextualizations into the session;

o Create a rubric for the program manager to use when observing Incredible Years sessions

that includes cultural contextualization pieces;

o Add items to the parent surveys administered after each session and to the parent 

satisfaction survey administered at the end of programming that expressly ask parents 
Center for Applied Research and Educational Improvement 

University of Minnesota 35 

whether they felt that their culture had been taken into account or that the terminology 

was understandable. 

The purpose of the grant was to test whether an evidence-based program that did not necessarily have 

explicit cultural aspects could be modified to include more culturally appropriate pieces (e.g., 

terminology, video vignettes, examples) and then show the same or better outcomes for the participants. 

This question fits nicely with the theory of change put forth by the SPPN, which posits that “...culture is 

the missing piece of the puzzle in the education process.” SPPN believes that partner programs should 

either be culturally-based at conception or should be adapted to include a cultural lens. In proposing the 

cultural contextualization of the Incredible Years curriculum, the question was “what would families need 

to make this more culturally appropriate and to feel connected to it?” Facilitators were then trained on 

INTERFACE and how to include cultural contextualization in the programming. The purpose of this 

grant was to validate the theory; however, because of the way it unfolded, the number of people 

participating in the cultural contextualization groups was lower than hoped. Project staff felt that the 

majority of the grant period was then spent on “getting the numbers,” rather than focusing on how best to 

refine the Hmong cultural contextualization and to develop the African American version. However, in 

the process, Incredible Years staff members reported that they learned several lessons: (1) adapting an 

evidence-based program is time-consuming work; (2) it takes time to build trust and rapport in 

communities, especially when you want to offer something new and test it at the same time; (3) 

recruitment strategies need to also be culturally contextualized; and (4) it may be best to partner with 

organizations that already serve the cultural groups your service is targeting, but do not currently offer the 

programming you provide. The test will be how Wilder and SPPN use the lessons they learned from 

participating in this project to continue to infuse culture into their programming. 
Center for Applied Research and Educational Improvement 

University of Minnesota 36 

REFERENCES 

Abidin, R. R. (1995). Parenting Stress Index (3rd ed.). Odessa, FL: Psychological Assessment Resources. 

Bermudez Parsai, M., Gonzalez Castro, F., Marsiglia, F. F., Harthun, M. L., & Valdez, H. (2011). Using 

community-based participatory research to create a culturally grounded intervention for parents 

and youth to prevent risky behaviors, Prevention Science, 12, 34-37.

Brenna, L. M., Shelleby, E. C., Shaw, D. S., Gardner, F., Dishion, T. J., & Wilson, M. (2013). Indirect 

effects of the Family Check-Up on school-age academic achievement through improvements in 

parenting in early childhood. Journal of Educational Psychology, 105(3), 762-773. 

Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child 

Psychology and Psychiatry, 38, 581–586. 

Gray, S. A. O., Heberle, A. E., & Carter, A. S. (2012). Social-emotional school readiness: How do we 

ensure children are ready to learn? Zero to Three, 33(1), 4-9. 

Greenberg, M. T., Kusch, C., & Mihalic, S. F. (1998). Blueprints for violence prevention, book ten: 

Promoting alternative thinking strategies (PATHS). Boulder: Center for the Study and Prevention 

of Violence, Institute of Behavioral Science, University of Colorado. 

Karcher-Ramos, M., MartinRogers, N., Mueller, D., & Mattessich, P. (2011). Saint Paul Promise 

Neighborhood: Summary of results from the 2011 community assessment and segmentation 

analysis. Saint Paul, MN: Wilder Research. 

Lau, A. S., (2006). Making the case for selective and directed cultural apadtations of evidence-based 

treatments: Examples from parent training, Clinical Psychology: Science and Practice, 13(4), 

295-310.

Lee, R. M., Vu, A. M., & Lau, A. S. (2013). Culture and evidence-based prevention programs. In F. 

Paniagua & A. M. Yamada (Eds.), Handbook of Multicultural Mental Health (2nd edition, pp. 

527-545). New York: Academic Press. 

McGilloway, S., Ni Mhaille, G., Bywater, T., Furlong, M., Leckey, Y., Kelly, P. et al. (2012). A 

parenting intervention for childhood behavioral problems: A randomized controlled trial in 

disadvantaged community-based settings. Journal of Consulting and Clinical Psychology, 80(1), 

116-127.

Niccols, A. (2009). Immediate and short-term outcomes of the “COPEing with Toddler Behaviour” parent 

group. Journal of Child Psychology and Psychiatry, 50(5), 617-626. 

Raver, C. C. (2002). Emotions matter: Making the case for the role of young children's emotional 

development for early school readiness. Social Policy Report, 16(3), 3-19. 

Rubin, K. H. (1998). Social and emotional development from a cultural perspective. Developmental 

Psychology, 34(4), 611-615. 
Center for Applied Research and Educational Improvement 

University of Minnesota 37

Schonert-Reichl, K. A., Smith, V., Zaidman-Zait, A., & Hertzman, C. (2012). Promoting children’s 

prosocial behaviors in school: Impact of the “Roots of Empathy” program on the social and 

emotional competence of school-aged children. School Mental Health, 4(1), 1-12. 

Squires, J., Bricker, D., & Twombly, E. (2002). Ages and Stages Questionnaires: Social-Emotional: A 

parent-completed, child-monitoring system for social-emotional behaviors. Baltimore: Paul H. 

Brookes Publishing Co.

Squires, J., Twombly, E., Bricker, D., & Potter, L. (2009). ASQ 3 user’s guide. Baltimore, MD: Paul H.

Brookes Publishing Co.

Saint Paul Promise Neighborhood. (August 2015). Impact document (downloaded from 

https://www.wilder.org/Community-Leadership/Saint-Paul-Promise-Neighborhood/Pages/OurImpact.aspx).

Weissberg, R. P., Kumpfer, K. L., & Seligman, M. E. P. (2003). Prevention that works for children and 

youth. American Psychologist, 58(6/7), 425-432.

Winter, L., Morawska, A., & Sanders, M. R. (2012). The effect of behavioral family intervention on 

knowledge of effective parenting strategies. Journal of Child and Family Studies, 21, 881-890.

Yang, M. (November 25, 2015). Parent Education Summaries (report prepared for Judy Ohm, Director of 

the Child Development Center). St. Paul, MN: Amherst H. Wilder Foundation. 

Zins, J., Bloodworth, M., Weissberg, R., & Walberg, H. (2004). The scientific base linking social and 

emotional learning to school success. In J. Zins, R. Weissberg, M. Wang, & H. Walberg (Eds.), 

Building academic success on social and emotional learning: What does the research say? (pp. 

3-22). New York: Teachers College Press.
Center for Applied Research and Educational Improvement 

University of Minnesota 38 

APPENDICES 

Appendix A: Background on the Development of the Hmong Cultural 

Contextualization 

Prior to receiving funding from the Social Innovation Fund, Wilder worked with University of Minnesota 

Professor Richard M. Lee and his colleagues to develop and research a culturally contextualized version 

of the Incredible Years program for Hmong families.24 The cultural contextualization was created using a 

respectful and collaborative process where parents who participated in the curriculum gave much input 

into and feedback concerning the process and the experience. Every topic in the curriculum was looked at 

through a Hmong cultural lens and contextualized appropriately. The process emphasized a conscious 

integration of community members’ needs and wants as well as their socioeconomic and cultural contexts 

(Bermudez Parsai, Gonzalez Castro, Marsiglia, Harthun, & Valdez, 2011). Additionally, as recommended 

by Lau (2006), Wilder utilized a dual approach to cultural contextualization. One component of the dual 

approach involved contextualizing content so that the contextualization accommodated the distinctive 

contextual factors related to the problem in the target Hmong community. The other component involved 

enhancing engagement such that contextualization increased engagement but did not undermine the 

outcomes obtained with the original intervention. 

A group of Hmong parents who completed Incredible Years served as members of an advisory group for 

the program. The input they provided helped ensure that the Incredible Years contextualization was 

especially responsive to the Hmong community. For instance, as a result of the advisory group’s 

recommendations, the curriculum was contextualized to accommodate the learning style of the Hmong 

parent participants. More specifically, rather than completing a topic during each session, Hmong parents 

advised that they preferred to start discussion of a topic in one session, have time to process it afterwards 

at home, and then revisit it with their thoughts and reactions in a subsequent session. Another example 

involved a change in symbols used in the curriculum. For example, the traditional curriculum uses a 

pyramid model that represents approaches to building parent/child relationships. The parent advisory 

group suggested that the image of a house (with a foundation, more rooms on the main floor, moving up 

to the attic) was more relevant and meaningful in their culture. 

Dr. Lee’s research highlighted three main themes and nine subthemes for the Hmong cultural 

contextualization of the Incredible Years program (Lee, Vu, & Lau, 2013). These themes reflect deeper 

structural changes that need to be taken into consideration when creating a culturally contextualized 

Incredible Years curriculum for Hmong families. These data-driven themes address: a) communication 

and learning styles and preferences of participants (e.g., narrative, indirect/vicarious learning, 

uninterrupted turn taking); b) Hmong cultural factors that inform parenting (e.g., multiple children and 

caregivers, cultural role expectations, traditional parenting and child development); and c) acculturative 

stresses that affect parenting (e.g., time demands, paradigm shift, parenting concerns). Parent feedback at 

the end of weekly sessions and at the end of the program was overwhelmingly positive in terms of 

delivery format and content. Parents also expressed a strong preference for a Hmong-only group format 

and recognized the need to target less acculturated, Hmong-language-only families whose need for parent 

training might be especially great. Moreover, Hmong parents who had participated in Incredible Years 

indicated that they wanted to receive more parent training, and some former participants stated that they 

had encouraged other Hmong families to participate.

 

24 This information about developing the Hmong cultural contextualization was initially provided in the final, 

approved SIF evaluation plan. 
Center for Applied Research and Educational Improvement 

University of Minnesota 39 

Appendix B: Wilder’s Incredible Years Program Descriptions 

Parents and Babies Program (10 weeks). Babies ages 0-12 months participate with their parents in this 

group. Childcare is provided for siblings by staff trained in the Incredible Years curricula. The Parents 

and Babies Program is a six-part program focused on helping parents learn to observe and read their 

babies' cues and learn ways to provide nurturing and responsive care including physical, tactile, and 

visual stimulation as well as verbal communication. The series includes: 

Part 1 - Getting to Know Your Baby (0-3 months) 

Part 2 - Babies as Intelligent Learners (3-6 months) 

Part 3 - Providing Physical, Tactile and Visual Stimulation 

Part 4 - Parents Learning to Read Babies' Minds 

Part 5 - Gaining Support 

Part 6 - Babies' Emerging Sense of Self (6-12 months) 

Parents and Toddlers Program (10 weeks). Parents of toddlers ages 1-3 years participate in this group. 

Childcare is provided for siblings. The Parents and Toddlers Program builds optimal parenting skills and 

consists of an eight-part program focused on strengthening positive and nurturing parenting skills. The 

toddler age child will join their parent(s) for “interaction time” at the end of each session. Each part 

builds on the previous. The series includes: 

Part 1 - Child-Directed Play Promotes Positive Relationships 

Part 2 - Promoting Toddler's Language with Child-Directed Coaching 

Part 3 - Social and Emotional Coaching 

Part 4 - The Art of Praise and Encouragement 

Part 5 - Spontaneous Incentives for Toddlers 

Part 6 - Handling Separations and Reunions 

Part 7 - Positive Discipline - Effective Limit Setting 

Part 8 - Positive Discipline - Handling Misbehavior 

Parents and Toddlers Attentive Parenting Program (6 weeks). Parents of toddlers ages 2-4 years 

participate in this group. Toddlers join their parents for “practice time” during each session. Childcare is 

provided for siblings. Similar to the 10-week group, this program builds optimal parenting skills and 

incorporates curriculum elements such as those described above that are focused on strengthening 

positive and nurturing parenting skills and building social, emotional, and developmental skills for school 

readiness. 

Early School Age (Preschool) Attentive Parenting Program (6 weeks). Parents of early school-age 

children (preschoolers) ages 3-5 years participate in this group. Children join their parents for “practice 

time” during each session. Childcare is provided for siblings. The program focuses on strengthening 

parenting skills and consists of components which build upon one another. The series includes: 

Part 1 - Strengthening Children's Social Skills, Emotional Regulation, and School Readiness 

Skills 

Part 2 - Using Praise and Incentives to Encourage Cooperative Behavior 

Part 3 - Positive Discipline - Rules, Routines, and Effective Limit Setting 

Part 4 - Positive Discipline - Handling Misbehavior 
Center for Applied Research and Educational Improvement 

University of Minnesota 40 

Appendix C: Incredible Years Parent Programs Cultural Contextualization 

Logic Model 
Center for Applied Research and Educational Improvement 

University of Minnesota 41 

Appendix D: Impact Evaluation Findings for the African American Cultural 

Contextualization 

PSI-SF Total Stress. For the Total Stress score, a score greater than or equal to 86 corresponds to a 

“clinical” score and a score lower than 86 corresponds to a “normal” score. This cutoff is represented as a 

dashed line in Figure D1. Figure D1 also shows the PSI-SF Total Stress scores for each parent (light gray 

lines) from pretest to posttest (13 parents met the criteria for inclusion). The downward slope of the 

heavier line (i.e., the mean change) shows that scores went down after participation in the program. The 

mean pretest score was 74.5 (SD = 21.6) and the mean posttest score was 62.0 (SD = 16.4). Cohen’s d

was 0.74, a moderately large effect. 

Changes in classification (i.e., clinical or normal) also occurred from pretest to posttest: 

 3 African American parents had PSI-SF Total Stress scores that were categorized as clinical at the 

pretest. Of these 3, 0 had clinical scores at the posttest and 3 had normal scores. 

 10 African American parents had scores that were normal at the pretest. Of these 10, 1 had a 

clinical posttest score and 9 had normal scores. 

 In summary, 3 African American parents improved in classification, 1 worsened, and 9 stayed the 

same. 

Figure D1. PSI-SF Total Stress (African American) 
Center for Applied Research and Educational Improvement 

University of Minnesota 42 

PSI-SF Parental Distress Subscale. Figure D2 shows results for the 13 parents who met the inclusion 

criteria on the Parental Distress subscale on the PSI-SF.25 The pretest mean was 28.2 (SD = 11.6) and the 

posttest mean was 23.7 (SD = 8.9). Cohen’s d was 0.56. 

The dashed line in Figure D2 represents the cut score for a classification of clinical (above the dashed 

line) or normal (below the line). For the Parental Distress subscale, a score greater than or equal to 33 

corresponded to a clinical score and a score lower than 33 corresponded to a normal score. The changes in 

classification (i.e., clinical or normal) from pretest to posttest were as follows: 

 5 African American parents had PSI-SF Parental Distress scores that were categorized as clinical 

at the pretest. Of these, 2 continued to have clinical scores at the posttest, while 3 had normal 

scores. 

 8 African American parents had pretest scores that were normal. Of these, 0 African American 

parents had clinical posttest scores, while 8 had normal scores. 

 In summary, 3 African American parents improved in classification, 0 worsened, and 10 

remained the same. 

Figure D2. PSI-SF Parental Distress (African American) 

 

25 Note that in some figures, the number of lines will not be the same as the number of scores represented, because 

some parents (PSI-SF) or children (SDQ) had overlapping pretest and posttest scores and trajectories. 
Center for Applied Research and Educational Improvement 

University of Minnesota 43 

PSI-SF Parental Child Dysfunctional Interaction Subscale. The mean pretest score on the Parental 

Child Dysfunctional Interaction subscale was 19.3 (SD = 6.1); the mean posttest score was 17.5 (SD = 

5.5) (see Figure D3). Cohen’s d was 0.26. 

For the Parent-Child Dysfunctional Interaction scale, a score greater than or equal to 26 corresponded to a 

clinical score and a score lower than 26 corresponded to a normal score. In terms of changes in 

classification category from pretest to posttest: 

 2 parents had PSI-SF scores that were classified as clinical at the pretest. Of these, 0 continued to 

have clinical scores at the posttest, while 2 had normal scores. 

 11 parents had scores that were normal at the pretest. Of these, 1 parent had a clinical posttest 

score, while 10 had normal scores. 

 To summarize, 2 African American parents improved in classification, 1 worsened, and 10 

remained the same. 

Figure D3. PSI-SF Parental Child Dysfunctional Interaction (African American) 
Center for Applied Research and Educational Improvement 

University of Minnesota 44 

PSI-SF Difficult Child Subscale. Figure D4 shows that, for the Difficult Child subscale, the pretest mean 

was 27.0 (SD = 9.1); the posttest mean was 20.8 (SD = 6.2). Cohen’s d was 0.95, which represents a large 

effect. 

For the Difficult Child subscale, a score greater than or equal to 33 corresponded to a clinical score and a 

score lower than 33 corresponded to a normal score. Again, there was not much change in classification 

category from pretest to posttest: 

 3 parents had PSI-SF Difficult Child subscale scores that were clinical at the pretest. Of these, 1 

continued to have a clinical score at posttest, while 2 had normal scores. 

 10 parents had scores that were normal at the pretest. Of these, 0 had clinical posttest scores, 

while 10 had normal scores. 

 In summary, 2 parents improved, 0 worsened, and 11 remained the same. 

Figure D4. PSI-SF Difficult Child (African American) 
Center for Applied Research and Educational Improvement 

University of Minnesota 45 

SDQ Total Difficulties Score. Figure D5 shows the scores on the SDQ for the 8 children with pretest and 

posttest SDQ scores (lightly shaded lines). The heavily shaded lines shows that the pretest mean was 11.9 

(SD = 6.2) and the posttest mean was 9.6 (SD = 3.8). Cohen’s d was 0.39. 

Total scores between 0 and 13 were considered normal, 14-16 were considered borderline, and 17-40 

were considered abnormal. The dashed line in Figure D5 represents the cutoff between an abnormal 

classification (above the dashed line) and a borderline or normal classification (below the line): 

 There were a total of 2 African American children with abnormal scores on the pretest. Both had 

normal posttest scores. 

 One child who was classified as borderline on the pretest had an abnormal posttest score. 

 There were a total of 5 children with a classification of normal at both pretest and posttest. 

 In summary, 2 children improved their classification, 1 worsened, and 5 remained the same. 

Figure D5. SDQ Total Difficulties Score (African American) 
Center for Applied Research and Educational Improvement 

University of Minnesota 46 

SDQ Emotional Problems Subscale. For this scale (see Figure D6), the pretest mean was 2.6 (SD = 1.7) 

and the posttest mean 2.1 (SD = 1.4). Cohen’s d was 0.25. 

Emotional problem scores between 0 and 3 were considered normal, a score of 4 was considered 

borderline, and scores of 5-10 were considered abnormal. Classification changes from pretest to posttest 

were as follows (again, the dashed line represents the cutoff above which scores are classified as 

abnormal): 

 1 child had an abnormal emotional problems score on the pretest, but a normal score on the 

posttest. 

 2 had borderline emotional problems scores on the pretest. Of these 2 children, 1 child had a 

normal posttest score and 1 had a borderline score. 

 5 children had normal scores on both the pretest and the posttest. 

 Overall, 2 children improved and 6 remained the same. 

Figure D6. SDQ Emotional Problems Subscale (African American) 
Center for Applied Research and Educational Improvement 

University of Minnesota 47 

SDQ Conduct Problems Subscale. Error! Reference source not found.Figure D7 shows that the mean 

pretest score for the SDQ Conduct Problems subscale was 2.5 (SD = 2.6) and the mean of the posttest was 

1.8 (SD = 1.8). Cohen’s d was 0.41. 

Conduct Problems subscale scores between 0 and 2 were considered normal, a score of 3 was considered 

borderline, and scores of 4-10 were considered abnormal. In terms of classification change from pretest to 

posttest: 

 3 African American children had abnormal Conduct Problem scores on the pretest. Of these 3 

children, 1 had a normal posttest score, 1 had a borderline posttest score, and 1 had an abnormal 

score. 

 No children had borderline scores on the pretest. 

 5 children had normal scores on the pretest. Of these, 4 continued to have normal scores on the 

posttest and 1 had a borderline posttest score. 

 Overall, the classification category for 2 children improved, for 1 child worsened, and for 5 

children the category remained the same. 

Figure D7. SDQ Conduct Problems Subscale (African American) 
Center for Applied Research and Educational Improvement 

University of Minnesota 48 

SDQ Hyperactivity Subscale. For the SDQ Hyperactivity subscale (see Figure D8), the pretest mean 

was 4.4 (SD = 2.6); the posttest mean was 3.9 (SD = 2.2). Cohen’s d was 0.42. 

Hyperactivity scores between 0 and 5 were considered normal, a score of 6 was considered borderline, 

and scores of 7-10 were considered abnormal. In terms of classification change from pretest to posttest: 

 2 children had abnormal scores on the pretest. Of these 2 children, 1 had a borderline posttest 

score, and 1 had an abnormal posttest score. 

 1 child had a borderline score on the pretest and a normal score on the posttest. 

 5 children had normal scores on the pretest and normal scores on the posttest. 

 In summary, 2 children showed an improvement in classification from pretest to posttest and 6 

remained the same. 

Figure D8. SDQ Hyperactivity Subscale (African American) 
Center for Applied Research and Educational Improvement 

University of Minnesota 49 

SDQ Peer Problems Subscale. On the Peer Problems Subscale for the African American cultural 

contextualization groups (see Figure D9), the mean of the pretest was 2.4 (SD = 1.4) and the mean of the 

posttest was 1.9 (SD = 1.1). Cohen’s d was 0.24. 

Peer Problems subscale scores between 0 and 2 were considered normal, a score of 3 was considered 

borderline, and scores of 4-10 were considered abnormal. Of the 8 children who had pretest and posttest 

scores, the following classifications were observed: 

 2 children had abnormal scores on the pretest; both had normal posttest scores. 

 2 had borderline scores on the pretest. On the posttest, 1 child had a borderline score and 1 had a 

normal score. 

 4 children had normal scores on the pretest. Of these, 2 had normal posttest scores and 2 had 

borderline posttest scores. 

 In summary, 3 children improved, 2 worsened, and 3 remained the same. 

Figure D9. SDQ Peer Problems Subscale (African American) 
Center for Applied Research and Educational Improvement 

University of Minnesota 50 

SDQ Prosocial Subscale. In contrast to the rest of the SDQ subscales, a higher score is more desirable 

than a lower score for the Prosocial subscale. Thus, in Figure D10, the area below the dashed line 

signifies a classification of abnormal and the heavier line shows movement from a lower score to a higher 

score, which indicates improvement from pretest to posttest on this subscale. On the Prosocial subscale, 

the pretest mean was 7.0 (SD = 2.1) and the posttest mean was 8.6 (SD = 1.3). Cohen’s d was 1.02, which 

is quite large and suggests a strong effect. 

Prosocial scores between 6 and 10 were considered normal, a score of 5 was considered borderline, and 

scores of 0 - 4 were considered abnormal. In terms of classification changes from pretest to posttest: 

 No children had abnormal scores on the pretest. 

 3 children had borderline scores on the pretest; they all had normal posttest scores. 

 5 children had normal scores on the pretest; all 5 had normal scores on the posttest. 

 Overall, 3 children improved and 5 remained the same. 

Figure D10. SDQ Prosocial Subscale (African American) 
Center for Applied Research and Educational Improvement 

University of Minnesota 51 

Appendix E: Impact Evaluation Findings for the Hmong Cultural 

Contextualization 

PSI-SF Total Stress. Figure E1 shows the PSI-SF Total Stress scores for each Hmong parent (lighter 

lines) from pretest to posttest (26 parents met the criteria for inclusion). The heavier line shows the mean 

score at pretest and posttest. The downward slope of the heavier line shows that scores went down after 

participation in the program (which is what is desired). The mean pretest score was 78.5 (SD = 23.7) and 

the mean posttest score was 71.5 (SD = 21.4). A paired t test was performed to determine if the difference 

was significant. There was no evidence of a significant change in Total Stress score from the pretest to the 

posttest (t(25) = 1.09, p = 0.28, Cohen’s d = 0.36). 

For the Total Stress score, a score greater than or equal to 86 corresponds to a “clinical” score and a score 

lower than 86 corresponds to a “normal” score. This cutoff is represented as a dashed line in Figure E1. 

Changes in classification (i.e., clinical or normal) also occurred from pretest to posttest: 

 9 Hmong parents had PSI-SF Total Stress scores that were categorized as clinical at the pretest. 

Of these 9, 3 continued to have clinical scores at the posttest and 6 had normal scores. 

 17 Hmong parents had scores that were normal at the pretest. Of these 17, 2 Hmong parents had 

clinical scores at the posttest and 15 had normal scores. 

 In summary, 6 Hmong parents improved in classification, 2 worsened, and 18 stayed the same. 

Figure E1. PSI-SF Total Stress (Hmong) 
Center for Applied Research and Educational Improvement 

University of Minnesota 52 

PSI-SF Parental Distress Subscale. Figure E2 shows results for the 26 parents who met the inclusion 

criteria on the Parental Distress subscale on the PSI-SF. The pretest mean was 26.8 (SD = 8.9) and the 

posttest mean was 24.8 (SD = 10.5). There was no evidence of change from pretest to posttest for the 

Hmong group on the Parental Distress subscale (t(25) = 0.80, p = 0.43, Cohen’s d = 0.22). 

The dashed line in Figure E2 represents the cut score for a classification of clinical (above the dashed 

line) or normal (below the line). For the Parental Distress scale, a score greater than or equal to 33 

corresponded to a clinical score and a score lower than 33 corresponded to a normal score. The changes in 

classification (i.e., clinical or normal) from pretest to posttest were as follows: 

 8 Hmong parents had Parental Distress subscale scores that were categorized as clinical at the 

pretest. Of these 8 Hmong parents, 3 continued to have clinical scores at the posttest and 5 had 

normal posttest scores. 

 18 Hmong parents had pretest scores that were in the normal range. Of these, 3 had clinical 

posttest scores and 15 had normal posttest scores. 

 In summary, 5 Hmong parents improved in classification, 3 worsened, and 18 remained the same. 

Figure E2. PSI-SF Parental Distress (Hmong) 
Center for Applied Research and Educational Improvement 

University of Minnesota 53 

PSI-SF Parental Child Dysfunctional Interaction Subscale. For Hmong parents, the mean pretest score 

on the Parental Child Dysfunctional Interaction subscale was 24.3 (SD = 8.7); the mean posttest score was 

21.5 (SD = 6.6) (see Figure E3). There was no evidence of change from pretest to posttest for the Hmong 

group on the Parental Child Dysfunctional Interaction subscale (t(25) = 1.29, p = 0.21, Cohen’s d = 0.42). 

For the Parent-Child Dysfunctional Interaction scale, a score greater than or equal to 26 corresponded to a 

clinical score and a score lower than 26 corresponded to a normal score. In terms of changes in 

classification category from pretest to posttest: 

 10 Hmong parents had PSI-SF Parental Child Dysfunctional Interaction subscale scores that were 

classified as clinical at the pretest. Of these 10 parents, 4 stayed in the clinical range at posttest, 

while 6 moved into the normal range. 

 16 Hmong parents had pretest scores that were normal. Of these, 2 had clinical posttest scores and 

14 had normal scores. 

 In summary, 6 Hmong parents improved in their classification from pretest to posttest, 2 

worsened, and 18 stayed the same. 

Figure E3. PSI-SF Parental Child Dysfunctional Interaction (Hmong) 
Center for Applied Research and Educational Improvement

University of Minnesota 54 

PSI-SF Difficult Child Subscale. Figure E4 shows that, for the Difficult Child subscale, the mean of the 

pretest was 27.4 (SD = 9.2) and the mean of the posttest was 25.3 (SD = 7.3). There was no evidence of 

change from pretest to posttest for the Hmong group on the Difficult Child subscale (t(25) = 0.98, p = 

0.34, Cohen’s d = 0.29). 

For the Difficult Child subscale, a score greater than or equal to 33 corresponded to a clinical score and a 

score lower than 33 corresponded to a normal score. Again, there was not much change in classification 

category from pretest to posttest: 

 8 Hmong parents had scores classified as clinical at pretest. Of these 8, 4 continued to have

clinical scores at the posttest, while 4 had normal scores.

 18 Hmong parents had scores that were normal at the pretest. Of these 18, none had posttest

scores in the clinical range and 18 remained in the normal range.

 In summary, 4 Hmong parents improved, 0 worsened, and 22 stayed the same.

Figure E4. PSI-SF Difficult Child (Hmong) 
Center for Applied Research and Educational Improvement 

University of Minnesota 55 

SDQ Total Difficulties Score. Figure E5 shows the scores on the SDQ for the 19 children with pretest 

and posttest SDQ scores (lightly shaded lines). The heavily shaded line shows that the pretest mean was 

10.1 (SD = 5.5) and the posttest mean was 7.9 (SD = 6.2). There was no evidence of change from pretest 

to posttest for the Hmong group on the SDQ Total Difficulties score (t(18) = 1.7, p = 0.11, Cohen’s d = 

0.57. 

Total Difficulties scores between 0 and 13 were considered normal, 14-16 were considered borderline, 

and 17-40 were considered abnormal. The dashed line in Figure D5 represents the cutoff between an 

abnormal classification (above the dashed line) and a borderline or normal classification (below the line): 

 There were a total of 3 Hmong children with abnormal scores on the pretest. Of the 3 Hmong 

children with abnormal pretest scores, 2 of these children moved from abnormal to borderline and 

1 remained at abnormal from pretest to posttest. 

 1 Hmong child was classified as borderline during the pretest, but moved to the normal score 

range on the posttest. 

 2 Hmong children had scores in the normal range on the pretest; both moved to borderline for the 

posttest. 

 13 Hmong children were classified as normal during the pretest and remained at normal for the 

posttest. 

 In summary, 3 Hmong children improved in classification, 2 worsened, and 14 remained the 

same. 

Figure E5. SDQ Total Difficulties Score (Hmong) 
Center for Applied Research and Educational Improvement 

University of Minnesota 56 

SDQ Emotional Problems Subscale. For this scale (see Figure E6), the pretest mean was 1.9 (SD = 1.9) 

and the posttest mean 1.3 (SD = 1.6). There was no evidence of change from pretest to posttest for the 

Hmong group on the Emotional Problems subscale (t(18) = 1.64, p = 0.12, Cohen’s d = 0.39). 

Scores on the Emotional Problems subscale between 0 and 3 were considered normal, a score of 4 was 

considered borderline, and scores of 5-10 were considered abnormal. Classification changes from pretest 

to posttest were as follows (again, the dashed line represents the cutoff above which scores are classified 

as abnormal): 

 2 Hmong children had an abnormal score on the pretest and both had borderline scores on the 

posttest. 

 1 child had a normal score on the pretest and a borderline score on the posttest. 

 16 Hmong children had normal scores on both the pretest and the posttest. 

 Overall, 2 Hmong children improved, 1 worsened, and 16 stayed the same. 

Figure E6. SDQ Emotional Problems Subscale (Hmong) 
Center for Applied Research and Educational Improvement 

University of Minnesota 57 

SDQ Conduct Problems Subscale. Figure E7 shows that the mean pretest score for the SDQ Conduct 

Problems subscale was 2.2 (SD = 1.6) and the mean posttest score was 1.8 (SD = 1.8). There was no 

evidence of change from pretest to posttest for the Hmong group on the Conduct Problems subscale (t(18) 

= 0.88, p = 0.39, Cohen’s d = 0.35). 

Conduct Problems subscale scores between 0 and 2 were considered normal, a score of 3 was considered 

borderline, and scores of 4-10 were considered abnormal. In terms of classification change from pretest to 

posttest: 

 3 Hmong children had abnormal Conduct Problems subscale scores on the pretest. Of these 3 

children, 1 had a normal posttest score, 1 had a borderline posttest score, and 1 had an abnormal 

posttest score. 

 6 Hmong children had borderline scores on the pretest. Of these, 2 had normal posttest scores, 2 

had borderline posttest scores, and 2 had abnormal posttest scores. 

 10 Hmong children had normal pretest and posttest scores. 

 To summarize, 4 Hmong children improved in their classifications, 2 worsened, and 13 remained 

the same. 

Figure E7. SDQ Conduct Problems Subscale (Hmong) 
Center for Applied Research and Educational Improvement 

University of Minnesota 58 

SDQ Hyperactivity Subscale. For the SDQ Hyperactivity subscale (see Figure E8), the pretest mean was 

3.9 (SD = 2.6); the posttest mean was 3.3 (SD = 2.7). There was no evidence of change from pretest to 

posttest for the Hmong group on the Hyperactivity subscale (t(18) = 1.01, p = 0.32, Cohen’s d = 0.27). 

Hyperactivity subscale scores between 0 and 5 were considered normal, a score of 6 was considered 

borderline, and scores of 7-10 were considered abnormal. In terms of classification change from pretest to 

posttest: 

 3 Hmong children had abnormal scores on the pretest. Of these 3 children, 1 had a normal 

posttest score and 2 had abnormal posttest scores. 

 2 Hmong children had borderline scores on the pretest. At posttest, 1 had a normal score and 1 

had a borderline score. 

 14 Hmong children had normal pretest scores. Of these 14 children, 1 had a borderline posttest 

score and the other 13 had normal scores. 

 In summary, 2 Hmong children improved in their classifications, 1 worsened, and 16 remained 

the same. 

Figure E8. SDQ Hyperactivity Subscale (Hmong) 
Center for Applied Research and Educational Improvement 

University of Minnesota 59 

SDQ Peer Problems Subscale. For the Peer Problems subscale (see Figure E9), the pretest mean was 2.1 

(SD = 1.4); the posttest mean was 1.6 (SD = 1.8). There was no evidence of change from pretest to 

posttest for the Hmong group on the Peer Problems subscale (t(18) = 1.07, p = 0.30, Cohen’s d = 0.34).

Scores on the Peer Problems subscale between 0 and 2 were considered normal, a score of 3 was 

considered borderline, and scores of 4-10 were considered abnormal. Of the 19 children who had pretest 

and posttest scores, the following classifications were observed: 

 3 Hmong children had abnormal scores on the pretest and posttest. 

 4 Hmong children had borderline scores on the pretest. All 4 had normal scores on the posttest. 

 12 Hmong children had normal pretest scores. Of these 12 children, 2 had borderline posttest 

scores and the rest (10) had normal scores. 

 In summary, 4 Hmong children improved, 2 worsened, and 13 remained the same. 

Figure E9. SDQ Peer Problems Subscale (Hmong) 
Center for Applied Research and Educational Improvement 

University of Minnesota 60 

SDQ Prosocial Subscale. In contrast to the rest of the SDQ subscales, a higher score is more desirable 

than a lower score for the Prosocial subscale. Thus, in Figure E10, the area below the dashed line signifies 

a classification of abnormal. In this case, if the treatment has a positive effect, the pretest mean should be 

lower than the posttest mean (and the line should slope up from pretest to posttest). However, for the 

Hmong group, the pretest mean was 7.7 (SD = 2.0), which was higher than the posttest mean (6.3; SD = 

3.8). This change was not significant (t(18) = 1.38, p = 0.19, Cohen’s d = 0.34). 

Prosocial subscale scores between 6 and 10 were considered normal, a score of 5 was considered 

borderline, and scores of 0-4 were considered abnormal. In terms of classification changes from pretest to 

posttest: 

 1 Hmong child had an abnormal score on the pretest. This child had a normal score on the 

posttest. 

 2 Hmong children had borderline scores on the pretest. Of these 2 children, 1 child had a normal 

score on the posttest and 1 had an abnormal score on the posttest. 

 16 Hmong children had normal pretest scores. Of these 16 children, 4 had abnormal posttest 

scores and the rest (12) had normal scores. 

 Overall, 2 Hmong children improved, 5 worsened, and 12 remained the same. 

Figure E10. SDQ Prosocial Subscale (Hmong) 
Center for Applied Research and Educational Improvement 

University of Minnesota 

 

 61 

Appendix F: Survey Results for the African American Cultural 

Contextualization 

Table F1. The Overall Program (African American) 

Considerably Worse Slightly worse The same Slightly Improved Greatly 

worse improved improved

My child's social 

and emotional 

skills are (n=12)

- - - 25% 50% 17% 8%

My child's preacademic skills for 

language, reading, 

and persistence at a 

task are (n=12)

- - - 33% 33% 8% 25%

My child's selfregulation and 

problem solving 

skills are (n=12)

- - - 25% 42% 17% 17%

Very 

pessimistic Pessimistic Slightly pessimistic Neutral Slightly optimistic Optimistic Very optimistic

My overall 

feelings about my 

personal progress 

at using the 

attentive and 

coaching parent 

skills are that I am 

(n=12)

- - - 25% 33% 42% -

Very 

inappropriate Inappropriate Slightly inappropriate Neutral Slightly appropriate Appropriate Greatly appropriate

I feel that the 

approach used to 

strengthen my 

child's social and 

emotional 

behaviors in this 

program is (n=12)

- - - 17% 17% 25% 42%

Strongly not Not Slightly not Neutral Slightly Recommend Strongly 

recommend recommend recommend recommend recommend

Would you 

recommend the 

program to a friend 

or relative? (n=12)

- - - 25% - 25% 50%

Very negative Negative Slightly 

negative Neutral Slightly positive Positive Very positive

My overall feeling 

about achieving 

my goal in this 

program for my 

child and family is 

(n=12)

8% - - 8% 8% 25% 50%
Center for Applied Research and Educational Improvement 

University of Minnesota 62 

Table F2. Teaching Format (African American) 

Extremely 

useless Useless Slightly useless Neutral Somewhat useful Useful Extremely useful

Content of information 

presented was (n=12) - 8% - 8% 17% 42% 25%

Demonstration of parenting 

skills through the use of 

video vignettes was (n=12)

- - 17% 8% 8% 42% 25%

Group discussion of 

parenting skills was (n=12) - - - 17% 17% 25% 42%

Practice of play and 

coaching skills at home 

with your child was (n=12)

- - 17% - 8% 42% 33%

Reading chapters from the 

Incredible Years or 

Incredible Toddlers book - - 9% 9% 9% 46% 27%

was (n=11) 

Weekly handouts (e.g., 

& others) were (n=12)

tips - - 8% 8% 17% 42% 25%

Use of practice or role 

plays during group 

sessions were (n=12)

- - 17% 25% - 33% 25%

Table F3. Specific Parenting Techniques (African American) 

Extremely 

useless Useless Slightly useless Neutral Somewhat useful Useful Extremely useful

Attentive Child-Directed 

Play (n=12) - - 8% 8% - 42% 42%

Academic and Persistence 

Coaching (n=12) - - 8% 8% - 42% 42%

Social Coaching (n=12) - - 8% 8% - 50% 33%

Emotional Coaching 

(n=12) - - 8% 8% - 50% 33%

Teaching Children to 

Problem Solve through 

Play (n=12)

- - 8% 8% - 42% 42%

Helping child control 

his/her anger (n=12) - - 8% 8% - 33% 50%

This Overall Group 

Techniques (n=12)

of - - 8% 8% - 25% 58%
Center for Applied Research and Educational Improvement 

University of Minnesota 63 

Table F4. Evaluation of Parent Group Leaders (African American) 

Very poor Poor

Slightly 

below Average

Slightly 

above High Superior

average average

I feel the leader's 

was (n=22)

teaching - - - 9% 14% 27% 50%

The leader's 

was (n=22)

preparation - - - 18% 5% 27% 50%

Extremely 

dissatisfied Dissatisfied Slightly dissatisfied Neutral Slightly satisfied Satisfied Extremely satisfied

Concerning the leader's 

interest and concern in 

me and my child, it [sic] 

was (n=20) 

- - - - - 45% 55%

Extremely 

unhelpful Unhelpful Slightly unhelpful Neutral Slightly helpful Helpful Extremely helpful

At this point, I feel the 

leader in the program was 

(n=20)

- - - - - 35% 65%

Table F5. Parent Group (African American) 

Very 

unsupportive Unsupportive Somewhat unsupportive Neutral Somewhat supportive Supportive Very supportive

I feel the group was 

(n=11) - - - 9% 18% 27% 46%

Very 

uninterested Uninterested Somewhat uninterested Neutral Somewhat interested Interested Very interested

Concerning other group 

members' interest in me 

and my child, I felt they 

were (n=10)

- - - 20% 10% 30% 40%

Highly 

unlikely Unlikely Somewhat unlikely Neutral Somewhat likely Likely Very likely

How likely is it that you 

will continue meeting 

with one or more of the 

parents in your group? 

(n=11)

18% - 9% 9% 9% 27% 27%

Table F6. Parent Group (Continued; African American) 

Item

I would like to keep meeting as a group (n=10)

Yes

20%

No

80%
Center for Applied Research and Educational Improvement 

University of Minnesota 64 

Table F7. Your Opinion (African American) 

Item Open-Ended Responses*

How could the program 

have been improved to 

help you more? (8 

people commented; 1 

said N/A; 1 said No 

comment.)

 Cover more emotional topics

 More topics

 The curriculum was aggressive and ridiculous

 Get the book. More sessions—ours was only 7 sessions

 It could have been a little more lengthy

 I don’t think the program can be improved. I already think highly of the program.

At this time do you feel 

the need for further 

parenting assistance? 

Please elaborate. (9 

people commented; 1 

said N/A; 1 said No 

comment.)

 Could always learn more

 Need to keep practicing the coaching methods we were taught

 Same

 Yes because I am still a young, learning parent and I get frustrated

 Yes, other aspects of parenting

 No 

 No

What did you see as the 

main benefit of the 

Attentive Parenting 

Program? (9 people 

commented; 1 said N/A;

1 said No comment.)

 A chance to have dedicated time to think about my parenting styles and technique.

 Everything was very helpful

 Helping my children learn problem-solving skills. Helping my child learn. Prosocial 

behaviors/skills

 Learning new coaching techniques

 Other parent perspectives, professional input

 The different parenting techniques when the child is at play.

 The thermometer

* Minor corrections were made to enhance readability, but the intent of the statement is preserved. 
Center for Applied Research and Educational Improvement 

University of Minnesota 65 

Appendix G: Survey Results for the Hmong Cultural Contextualization 

Table G1. The Overall Program (Hmong) 

Considerably

worse Worse Slightly worse The same Slightlyimproved Improved Greatlyimproved

My child's social 

and emotional - - 5% 16% 21% 42% 16%

skills are (n=19)

My child's preacademic skills

for language,

reading, and

persistence at a 

task are (n=19)

- - - 16% 32% 32% 21%

My child's selfregulation and - problem solving - 5% 21% 37% 21% 16%

skills are (n=19)

Very

pessimistic Pessimistic Slightlypessimistic Neutral

Slightly

optimistic Optimistic Veryoptimistic

My overall 

feelings about my

personal progress

at using the 

attentive and

coaching parent 

skills are that I am

(n=19)

- - - 21% 11% 42% 26%

Very

inappropriate Inappropriate Slightlyinappropriate Neutral

Slightly

appropriate Appropriate Greatlyappropriate

I feel that the 

approach used to

strengthen my

child's social and

emotional 

behaviors in this

program is (n=19)

- - - - 5% 53% 42%

Strongly not 

recommend

Not 

recommend

Slightly not 

recommend

Neutral

Slightly

recommend

Recommend

Strongly

recommend

Would you

recommend the 

program to a 

friend or relative?

(n=19)

- - - - 5% 32% 63%

Very negative Negative

Slightly 

negative

Neutral

Slightly

positive

Positive Very positive

My overall feeling

about achieving

my goal in this

program for my

child and family

is (n=19)

- - - - 11% 47% 42%
Center for Applied Research and Educational Improvement 

University of Minnesota 66 

Table G2. Teaching Format (Hmong) 

Extremely 

useless Useless Slightly useless Neutral Somewhat useful Useful Extremely useful

Content of information 

presented was (n=19) - - - - - 42% 58%

Demonstration of parenting 

skills through the use of 

video vignettes was (n=19)

- - - - 11% 42% 47%

Group discussion of

parenting skills was (n=19) - - - - - 37% 63%

Practice of play and 

coaching skills at home 

with your child was (n=19)

- - - - 11% 32% 58%

Reading chapters from the 

Incredible Years or 

Incredible Toddlers book - - - 11% 33% 33% 22%

was (n=9)

Weekly handouts (e.g., 

& others) were (n=19)

tips - - - 5% 16% 42% 37%

Use of practice or role 

plays during group 

sessions were (n=19)

- - - 11% 16% 37% 37%

Table G3. Specific Parenting Techniques (Hmong) 

Extremely 

useless Useless Slightly useless Neutral Somewhat useful Useful Extremely useful

Attentive Child-Directed 

Play (n=19) - - - - - 47% 53%

Academic and Persistence 

Coaching (n=19) - - - - - 42% 58%

Social Coaching (n=19) - - - - 5% 32% 63%

Emotional Coaching 

(n=19) - - - - 11% 16% 74%

Teaching Children to 

Problem Solve through 

Play (n=19)

- - - - 5% 37% 58%

Helping child control 

his/her anger (n=19) - - - - 5% 37% 58%

This Overall Group 

Techniques (n=19)

of - - - - - 26% 74%
Center for Applied Research and Educational Improvement 

University of Minnesota 67 

Table G4. Evaluation of Parent Group Leaders (Hmong) 

Very poor Poor

Slightly 

below Average

Slightly 

above High Superior

average average

I feel the leader's 

was (n=38)

teaching - - - 5% 8% 37% 50%

The leader's 

was (n=37)

preparation - - 3% - 11% 30% 57%

Extremely 

dissatisfied Dissatisfied Slightly dissatisfied Neutral Slightly Satisfied Satisfied Extremely satisfied

Concerning the leader's 

interest and concern in me 

and my child, it [sic] was 

(n=37)

- - - 3% 5% 27% 65%

Extremely 

unhelpful Unhelpful Slightly unhelpful Neutral Slightly helpful Helpful Extremely helpful

At this point, I feel the 

leader in the program was 

(n=37)

- - - - - 35% 65%

Table G5. Parent Group (Hmong) 

Very 

unsupportive Unsupportive Somewhat unsupportive Neutral Somewhat supportive Supportive Very supportive

I feel the group 

(n=18)

was - - - - 6% 50% 44%

Very 

uninterested Uninterested Somewhat uninterested Neutral Somewhat interested Interested Very interested

Concerning other group 

members' interest in me 

and my child, I felt they 

were (n=18)

- - - - 11% 50% 39%

Highly 

unlikely Unlikely Somewhat unlikely Neutral Somewhat likely Likely Very likely

How likely is it that you 

will continue meeting 

with one or more of the 6% 6% - 22% 17% 17% 33%

parents in your group? 

(n=18)

Table G6. Parent Group (Continued; Hmong) 

Item Yes No

I would like to keep meeting as a group (n=16) 13% 88%
Center for Applied Research and Educational Improvement 

University of Minnesota 68 

Table G7. Your Opinion (Hmong) 

Item Open-Ended Responses*

How could the program 

have been improved to 

help you more? (13 

people commented)

 Nothing much.

 The program was really helpful.

 Everything's good.

 Overall the program itself was very helpful. It taught me to be more active in my baby sister’s

early childhood as she grows up and is learning at the same time.

 I would like it if the leader was more prepared and up to date with events outside of days that 

we meet in class. Don't wait for the last minute to tell us about outside events, or have to tell 

us about the events when the event ends on the day of.

 I think if the videos were translated into the Hmong language it would be better for our 

group.

 I think if the program was broken out longer, so we can learn slowly, it would be more 

helpful, because I understand that there are some parents that take a longer time to learn all 

the information.

 Less content and objectives to cover [each] session so there’s more time for parents to 

discuss, reflect and learn.

 Maybe we should have more topics added to the class.

 The program can improve with more families, however it’s understandable that families are 

busy, as time goes on there are less families showing up.

 I still think the program is geared too much on the 1 parent and 1 child ratio. Although this 

format may be useful in measuring results, it does not mirror what parents experience in the 

home with many children.

 Let the kid and parent play as a group to see if they can see what the child likes in class

 Kuv xav thov kom hais lus hmoob (Translation: “I would like there to be more Hmong 

spoken/used.”)

At this time do you feel 

the need for further 

parenting assistance? 

Please elaborate. (14

people commented)

 Nothing much.

 None

 Not sure

 No (4 people said “No”)

 None at all, but it would be nice. The program was very helpful.

 No, at this time, I think everything that was taught to us is good enough.

 No. The concepts taught during incredible years were very good concepts.

 Everything's good, helping one another being good parents.

 The techniques that were taught over the last month were very insightful and very helpful. 

The handouts were very helpful at home and now it's work that needs to be, but the program 

was very helpful and the group leaders did a great job!

 Yes. I feel that I need more help with having to be able to tell my child to do something or 

stop doing something for me when being asked to do so.

 I think that if I had the opportunity to speak or share more on my challenges in the home with 

my children then I would be better able to utilize lessons from this program to help my 

family.

What did you see as the 

main benefit of the 

Attentive Parenting 

Program? (17 people 

commented)

 I think the videos are a great way to teach, that way we have a visual to go off of.

 Hand out Assignments, videos, and examples were very beneficial. Everything else, and 

learning about your children's behavior was a good thing to know and how to resolve it.

 Maybe to understand the child's emotions.

 The main benefit is to let my child do things on her own and make her own decisions.

 Not only did it teach me to be a better role model for my sister, it also taught me to be more 

active and bond with her more. And I realize that the handout/ tips very helpful.

 Followings rules and bonding with child.

 Being able to understand my child more instead of just wanting things my way.

 Learning about how to know your child's emotion. Getting to know your child well.
Center for Applied Research and Educational Improvement 

University of Minnesota 69 

Item Open-Ended Responses*

 Learning developmental milestones about my child's growth that I wouldn't otherwise have 

known.

 The main benefit of the program is as a parent we can lead our kids in a positive way.

 Help me be a better parent and teach my kids to be better kids

 The main benefit I got from this program was not to only raise a child, but so that I can 

understand and know myself as a parent.

 The program was very helpful and at the same it taught me to grow as well; while focusing 

on [child’s] needs too, to grow and develop at the same time.

 The main benefit was the ability to have others support me while I take the time to learn these 

lessons. Often times in the home, I feel that all the responsibility is on myself to [sic] my 

spouse.

 Very helpful

 Ntau yam. (Translation: “Many things.”)

 Kev sib pab thiab sib raug zoo (Translation: “Sharing and relationship building”)

*Minor corrections were made to enhance readability, but the intent of the statement is preserved. 
Center for Applied Research and Educational Improvement 

University of Minnesota 70 

Center for Applied Research and Educational Improvement 

University of Minnesota 

1954 Buford Ave., Suite 425, St. Paul MN 55108 

Phone: 612-624-0300 • Fax: 612-625-3086 • carei@umn.edu