Research and Evidence Webinar Series:

November 17, 2020

Youth Interventions 

that Work

1
We’ll get started in just a 

couple of minutes.

Please be advised that 

there is no dial-in for this 

webinar; all audio is 

provided directly through 

the Adobe Connect 

platform.

Welcome!

Youth Interventions that Work

2
Katy Hussey-Sloniker

MBA, MSW, LGSW

Office of Research and Evaluation, AmeriCorps

Welcome!

Youth Interventions that Work

3
Presentations Today

4

Project: One Child at a Time Project: Shape NC

Project: G.O.A.L.S. Project: Great Families 2020

Social Innovation Fund Showcase: Youth Interventions that Work
Introductory Remarks

Youth Interventions that Work

5

Dr. Lily Zandniapour

Research and Evaluation Manager

Office of Research and Evaluation, AmeriCorps
Social Innovation Fund

How It Works

6

 
Youth Interventions that Work

Dr. Shelley Osborn, ICF

Dr. Susan Walsh, Friends of the Children

November 17, 2020
THE POWER OF ONE
HOW WE SELECT and INVITE CHILDREN AND FAMILIES

SCHOOLS FOSTER CARE COMMUNITY PARTNERS
SIX CORE ELEMENTS

1. Children facing the greatest obstacles

2. Professional mentors (Friends)

3. Commit for the long haul

4. Individualized and intentional

5. Home, school, community

6. Evaluate, measure, improve
MULTI-SITE RANDOMIZED 

CONTROL TRIAL 

Child Study (2007-2014)

Participant ages 5-11

Findings:

• More Prosocial Strengths

• Less Externalizing Behavior

• Less Trouble in School

• Positive Parent Perception of Behavior

Young Adult Study (2020-2025) 

Participant ages 19 and 21

Outcome Aims: 

• Three Long-term outcomes

• Social Capital

• Self Sufficiency
SIF AS CATALYST FOR SCALING 
SIF AS CATALYST FOR SCALING 
SIF EVALUATION DESIGN

• Implementation Study

• Child Welfare Study

• School Study

• Pilot Caregiver Study
KEY EVALUATION LESSONS LEARNED

• Implementation Study – The process is the most important part of the 

journey.

• School and child welfare administrative data analyses – Make sure the 

juice is worth the squeeze.

• Caregiver voice – Our families’ voice is priceless and setting the course 

for our future.
CAREGIVER VOICE – OUR 2GEN JOURNEY

Parents/Caregivers perceive their child:

• Improved their child’s Social-Emotional Skill development

• Liked to go to school more

• Was more hopeful and optimistic

Parents/Caregivers reported:

• Improved problem-solving within the family

• Improved ability to handle the day-to-day parenting demands

• Connections to educational assistance, mental health resources, child-care and 

cultural/health activities.
SIF EVAL AS A CATALYST FOR CHANGE
SIF EVAL AS A CATALYST FOR CHANGE
Lessons Learned in Shaping Healthy Starts for 

Young Children 

Stephania Sidberry, MPH

North Carolina Partnership for Children

November 17, 2020
Shape NC Program Team 

Stephania Sidberry, 

Shape NC Program 

Manager Courtney LattaSosebee, Shape NC 

Implementation 

Coach 

Angela Lewis, 

Shape NC 

Implementation 

Coach 
Shape NC History

 Shape NC: Healthy Starts for Young Children is a partnership between 

The Blue Cross and Blue Shield of North Carolina Foundation 

(BCBSNCF) and The North Carolina Partnership for Children, Inc. 

(NCPC). 

 Created in 2010 to increase the number of children starting 

kindergarten at a healthy weight and ready to learn.

 In 2016 received funding from CNCS to conduct a rigorous evaluation 

of the program
Expert 

Partners

 Be active Kids

 Natural Learning Initiative

 Go NAPSACC

 Farm to ECE

Learning 

Spaces

 Peer-to-Peer Learning Collaboratives

 Technical Assistance to Child Care Centers

 Coaching for Technical Assistants 

 Online Resources

 Conferences

Funding  ...to Local Partnerships

 ...to Child Care Centers
Methodology
Research Questions

1. What is the impact of Shape NC on the number of health and nutrition best 

practices implemented in participating child care sites receiving one or two 

years of treatment relative to comparison sites?

2. What is the impact of Shape NC on the physical activity of children in 

participating child care sites receiving one or two years of treatment relative 

to children in comparison sites? 

3. What is the impact of Shape NC on the percentage of participating children 

at a healthy weight as measured by child body mass index (BMI) status relative 

to children in comparison sites? 
Outcomes

RS1: There was no significant impact of Shape NC on the number of health and nutrition 

polices and best practices after one or two years of implementation. 

RS2: After two years, children at all 20 treatment sites engaged in indoor free play, an 

increase from one site at baseline (p=.00). 

After two years, the number of sites in which children engaged in structured indoor 

play/organized games increased from eight to 17 sites es (p=.01). 
RS2: After two years of treatment, the number of centers in which children 

participated in outdoor structured play/organized games increased from four to 17 

(p=.00). 

RS3: After two years of treatment, the percentage of children in the healthy weight 

category decreased significantly from 73.8 percent at baseline to 65.7 percent 

(p=0.02) while the percentage of children in the obese category increased 

significantly from 9.5 percent at baseline to 15.2 percent (p=0.02).

Outcomes
Expert 

Partners

 Be active Kids

 Natural Learning Initiative

 Go NAPSACC

 Farm to ECE

Learning 

Spaces

 Peer-to-Peer Learning Collaboratives

 Technical Assistance to Child Care Centers

 Coaching for Technical Assistants* (targeting scale) 

 Online Resources

 Conferences

Funding  ...to Local Partnerships

 ...to Child Care Centers

The G.O.A.L.S. 

project: 

A Social Innovation Fund 

Initiative by United Way for 

Southeastern Michigan
Acknowledgements 

Jeff Miles, 

Shauna Aitson, 

Maya Satterwhite, 

Alice Audie-Figueroa, Melanie 

Gill

“The Social Innovation Fund (SIF) was a program of the Corporation for National and Community Service that received funding from 2010 to 2016. 

Using public and private resources to find and grow community-based nonprofits with evidence of results, SIF intermediaries received funding to award 

subgrants that focus on overcoming challenges in economic opportunity, healthy futures, and youth development. Although CNCS made its last SIF 

intermediary awards in fiscal year 2016, SIF intermediaries will continue to administer their subgrant programs until their federal funding is exhausted.”

Agency Managers, Clinicians, 

Staff, and Families

Randi Burlew, 

Heather Hirsch
The G.O.A.L.S program
Agencies Implementing GOALS
Timeline

Full 

Programming 

Started

Pilot 

Started

Pilot 

Concluded

Final 

Enrollment 

Date 

Data 

Collection 

Ends 
Who did we 

serve? 
Participant Health Factors

86% Medicaid
FCU Feedback/Goal 

Setting Session

Family participates in 

agency services that they 

seek out

Referrals are made that 

align with the goals set by 

the family

Family participates in 

services/pathways to meet 

goals

Comparison Intervention

Assessments FCU Initial Interview + 

Assessments

6 & 12 month follow-up 

Assessments, Feedback

6 & 12 month follow-up 

Assessments

Family given a brief 

feedback form and a list of 

resources
Domains of Family Life Assessed
Comprehensive Assessments, Feedback, and Goal Setting

INTERVENTION GROUP FEEDBACK FORM INTERVENTION GROUP FEEDBACK FORM
Implementation Example

Get child to 

try more foods FEAST

Child starting 

accepting a variety 

of new foods

Get daughter 

(older child) 

ready for 

school

Parenting Pathway 

and Tips

Daughter attending 

school, no 

separation anxiety

Apply for food 

stamps or 

other type of 

assistance

Food vouchers at 

the agency, “211”

Went to social 

services, applied 

for, and began 

receiving food 

stamps

GOAL REFERRAL

STATUS AT 

FOLLOWUP 
Intervention Actions and Supports

TOP 10 GOAL CATEGORIES • TOP 10 REFERRALS
Primary “Pathways” 

https://www.oaklandfamilyservices.org/early-childhood-services

https://unitedwaysem.org/get-help/community-resources/ready4k/

https://www.nkfm.org/communities-families/diabetes-prevention-program

http://www.michildrenshealth.org/

https://unitedwaysem.org/blog/lets-feast-united-way-program-empowers-parents-toraise-healthy-eaters/
Key Outcomes

Health Care

Random-effects regression models estimating time by treatment group effects

For ordinal risk variables logistic random-effects models estimated likelihood 

of risk to no-risk categories. 
Family 

Impact of 

COVID-19
COVID-19 Survey Results 

(380 respondents)

COVID-19 Exposure

• 4% COVID Positive

• 11% immediate 

family members 

COVID Positive

• 40% extended family / 

close friends COVID 

Positive

• 18% have had a family 

member or close 

friend die from 

COVID-19

Risk Mitigation 

• 94% wore a mask 

when outside the 

home

• On average, Left 

home 3.5 days per 

week, had close 

contact w/ 7-8 

people, 

• 20% had to leave 

home for work

• 55% left home for 

groceries or shopping 

Income

• 72% decrease in 

income (31% lost 

their income)

• 23% moderate to 

severe impact on 

food access

• 45% moderate to 

severe increase in 

stress
COVID IMPACT: March – August 2020

Families 

were 

more 

at risk 

for

Inadequate Health Care

Inadequate Employment

Decreased Family Social Connections

Being Disconnected from the Community

Parenting Challenges 

Mental Health Challenges
The Positive Takeaway 

Families in the 

G.O.A.L.S Program 

were 1.8 – 3.5 times 

less likely to show an 

increased risk over 

time after April 30, 

2020 in areas of 

• Mental Health

• Parenting 

• Family Connections

• Health Care Access
Great Families 2020 - A 2Gen Approach

Lessons from the Field

11/17/20
• Whitney Fields, Senior Program Officer

• Denise Luster, VP Impact Research & Analytics

• Stephanie Fritz, Director of Strategic Research & 

Analytics

• Breanca Merritt, Director of the Center for 

Research on Inclusion and Social Policy

Presenters

48
The Two-Generation Approach

49
Key Components of Two-Gen

50
Great Families 2020 Subgrantees

GF2020 Program Details

 

3.5 years of subgrants 

to implement 2Gen 

Programs and 

Services for Families 

Subgrantee Autonomy

1:1 Program Officer Support

Serving Whole Family Unit

5 target areas

8 Subgrantees

 

 

 
Theory of Change

52

Families will be 

financially stable 

Children will enter 

kindergarten ready 

to learn

Parents will 

acquire 21st

century skills & 

credentials & 

career-track jobs

Children & parents 

will be physically 

and mentally 

healthy

Families will have 

formal & informal 

support networks
The Indiana Standards Tool for Alternate Reporting of Kindergarten 

Readiness (ISTAR-KR) tool assessed kindergarten readiness and 

overall child development. 

• Free/Web and paper administration options available

• Used from infancy to first grade to assess a child’s developmental 

skills in five domains: physical, personal care, socio-emotional skills, 

English-language arts, and math. 

• Age-based and benchmarked

53

Youth Evaluation Tool: 

ISTAR-KR
Student Identification Numbers (STNs) 

not collected by subgrantees due to 

FERPA regulations

335 youth records submitted to IDOE for 

matching, only 22 records matched

(6.6% match rate) 

Data Sharing Agreement with Indiana 

Department of Education 

(Spring 2018)

54

Evaluation Barriers: 

Data Collection and 

Data Sharing Agreements

State 

ISTARKR Data

GF2020 

Youth 

PII Data

ECE 

Outcomes
Implementation Challenges Impact on Evaluation

Only web-based 1-year waiver for providers with no computer 

or internet 

Annual assessment

ISTAR-KR was administered and updated 

throughout the year, resulting in the ability to 

track improvements due to high quality 

childcare.

Population-level assessment – data not 

reported on individuals, but county 

populations

Inability to track growth in GF2020 children. 

55

Evaluation Barriers: 

Policy Changes

In 2019, Indiana State Board of Education decommissioned 

ISTAR-KR and replaced it with Kindergarten Readiness Indicators 

(KRI).
ECE Attendance ISTAR-KR Parenting 

Outcomes

56

Youth Evaluation Outcomes

• Since October 2017, 70% 

of GF2020 youth 

(n=1,121) were enrolled 

in high-quality childcare 

on average 166 days.

22 youth were matched with 

IDOE records.

At the time of program exit:

• 32% of children had 

mastered all functional 

performance threads 

and 95% had mastered 

at least half.

• 18% mastered all 

Math/ELA threads; 55% 

had mastered at least half.

• Parents reported that their 

perception of their own 

understanding of child 

development and parenting 

improved significantly over 

the course of GF2020 

enrollment.

• This change was significant 

over all three time periods: 

from baseline to first follow-up 

(p<.001), baseline to second 

follow-up (p<.001), and 

baseline to third follow-up 

(p<.05). 
Data Sharing

• PII in data sharing 

agreements may not be 

enough for student 

matching. 

• Data sharing 

agreements should 

include STNs if possible.

Partnerships

• Creation of an advisory 

board to include 

membership with 

Indiana Department of 

Education.

• Formalized partnerships 

between subgrantees 

and ECE providers to 

improve attendance data 

and youth identification 

data.

Policy Implications

• State policy changes 

can directly impact local 

programming evaluation.

GF2020 Early Lessons Learned
Closing Remarks

Youth Interventions that Work

58

Katy Hussey-Sloniker

MBA, MSW, LGSW

Office of Research and Evaluation, AmeriCorps
Contact Information

59

Friends of the Children

Project Lead: Susan Walsh swalsh@friendsofthechildren.org

North Carolina Partnership for Children

Project Lead: Stephanie Sidberry ssidberry@smartstart.org

United Way for Southeastern Michigan

Project Lead: Melanie Gill Melanie.Gill@liveunitedsem.org

United Way of Central Indiana

Project Lead: Whitney Fields Whitney.Fields@uwci.org

Office of Research and Evaluation

Katy Hussey-Sloniker: khussey-Sloniker@cns.gov

Lily Zandniapour: lzandniapour@cns.gov
Questions?

Dr. Andrea Robles

Research and Evaluation Manager, 

Office of Research and Evaluation, AmeriCorps

Q&A

Youth Interventions that Work

60

Visit nationalservice.gov/ORE for past webinars, research and evaluation 

resources, reports, and more!