Colorado Parent and Child Foundation

Social Innovation Fund

Evaluation Report

December 2014

Denver, Colorado 

Prepared by

The Butler Institute for Families 

Graduate School of Social Work 

University of Denver 
Table of Contents 

Executive Summary.............................................................................................................................................. 1 

Methods .................................................................................................................................................................... 4 

Data analysis ....................................................................................................................................................... 5 

Programs .............................................................................................................................................................. 5 

Results ....................................................................................................................................................................... 6 

Program and Family Characteristics ..........................................................................................................6

Child Literacy & Communication ...............................................................................................................14

Parent Self-rated Knowledge and Skills ..................................................................................................18

Home Literacy Activities ...............................................................................................................................21

Interactive Reading Techniques ................................................................................................................22

Feedback from staff ........................................................................................................................................24

This study was made possible by the Mile High United Way Social Innovation Fund, 

Denver, Colorado.
P a g e | 1

Executive Summary 

The Colorado Parent and Child Foundation (CPCF) administers two evidence-based early childhood 

home visiting programs that aim to accomplish the vision that parents are their child’s first and most 

influential teachers who prepare children for success in school and life. The two programs—Parents 

as Teachers (PAT) and Home Instruction for Parents of Preschool Youngsters (HIPPY)—

together are administered to families of children age birth through kindergarten entry in 36

programs throughout Colorado. Parents as Teachers also serves women throughout pregnancy. 

In 2012, the Colorado Parent and Child Foundation (CPCF) began participating in the Mile High 

United Way (MHUW) Social Innovation Fund (SIF). The SIF, managed by MHUW, aims to increase 

the evidence-base of early childhood literacy programs in Colorado by funding multiple childserving organizations to implement rigorous programming and evaluation over a 5-year period. 

The CPCF participated in SIF for 2 years (2012-2014), providing eight PAT and HIPPY program sites 

resources to expand capacity by 33%, ultimately serving 251 more children. 

This report is based upon work supported by the Social Innovation Fund (SIF), a key White House 

initiative and program of the Corporation for National and Community Service (CNCS). The Social 

Innovation Fund combines public and private resources to grow the impact of innovative, 

community-based solutions that have compelling evidence of improving the lives of people in lowincome communities throughout the United States. 

The seven PAT programs and one HIPPY program funded through CPCF-SIF aimed to reach more 

families in need, and to help strengthen the evidence-base for early childhood literacy programs in 

Colorado. Participating PAT programs collected information from parents about their home literacy 

activities, children’s language and communication skills, and experiences receiving home visiting. 

These data, combined with parent educator accounts, director interviews and focus groups, and 

administrative program information, are used to describe the impact of the Social Innovation Fund 

in these communities through this report. Evaluation was conducted by the Butler Institute for 

Families at the University of Denver beginning in 2014. 

 

Executive Summary. Overall, programs reported achieving what they hoped to achieve with the 

CPCF-SIF grant. Programs reported increasing capacity to serve new families by hiring new staff, 

adding programs or program components, and expanding services to new communities or families. 

Programs well-exceeded the goal of increasing capacity to serve 184 more families by serving 

an additional 251 families within just 2 years of receiving the CPCF-SIF award. 

Infants (age 8 to 15.99 months) in the study appear developmentally “on-track” related to their 

early childhood language development. Infants score on average or within expected ranges on the 

two developmental measures used in the study—the MacArthur Communicative Development 

Inventories (CDI) and the Ages and Stages Questionnaire, Communication Subscale (ASQ). 

However, toddlers (age 16 to 38 months) have high rates of ASQ Communication concerns, 

indicating the need for referral for further developmental assessment. Rates of ASQ concerns 

among toddlers in the CPCF-SIF study were twice the rate expected among children in the general 

population, and four to five times higher at the follow-up time point compared with other children. 

Boys were four times more likely to have ASQ scores in the concern-range compared with girls. 

The following summarizes other primary findings of the CPCF-SIF study. 
P a g e | 2

Characteristics of families served 

 Across the seven PAT programs, 1,143 children were served, of which 251 (24%) were 

served through the SIF program specifically. This well-exceeds the goal of serving 184 more 

children through SIF. 

 Most children who received services were infants (n=375; 33%) or toddlers (n=410; 37%). 

Services were also provided to 116 expectant mothers (10%). 

 For 448 children, Spanish is their primary language. 

 A large number of families (n=848) are two-parent households. 

 One-third of parents have less than a high school education, well above the 12% of adults 

age 25 and older in the U.S. who have less than a high school education (U.S. Census, 2011). 

 Parent educators are diverse; 67% are Hispanic or Latino ethnicity, and 56% reported 

“bilingual” English-Spanish as their primary language. 

 40% of parent educators have a bachelor’s degree and 6% have a master’s degree. 

Child outcomes 

 Infants appear to be on-course in language development. No infants had ASQ 

Communication scores that indicated the need for referral to early intervention services, 

and vocabulary comprehension and word production were also comparable to other 

children. 

 Early toddlers (16-30 months), however, showed the most need, and ASQ concern rates 

among both early and late toddlers (30.1-38 months) at study intake were two times the 

rate or more observed among other children. Among late toddlers, 25% had ASQ 

Communication scores in the concern-range at follow-up, compared with 2.5% of children 

in the general population. 

 Early toddlers also showed below-average word production; 42% of early toddlers were at 

or below the 25th percentile in word production at intake to the study. 

 The high ASQ concern rate was concentrated among boys. Concern rates among boys were 

four times higher than girls at study intake. 

 More frequent home literacy activities were associated with lower rates of ASQ 

Communication concerns. In other words, parents who reported fewer home literacy 

activities were three times more likely to have a child with ASQ concerns compared with 

parents who reported more frequent home literacy activities. 

Parents 

 Parents are confident in their skills and knowledge related to child development, both at the 

time of study enrollment and follow-up assessments. Parents recognize the importance of 

well-child visits, healthy physical activity for children, and healthy diet. Parents reported 

slightly lower ratings of knowledge and skills related to preparing healthy foods and 

preventing obesity in their children. 
P a g e | 3

 There was a slight increase in parent’s reports of being knowledgeable about where to find 

community resources at follow-up. 

 Parents reported a significant increase in talking with children about books over time. 

 Parents reported significant increases over time in several other literacy activities, such as 

talking about stories without words, talking about and writing their child’s name, looking at 

and talking about shapes, and encouraging their child to read along with them, but the 

increases were likely due to children “aging into” these activities over time. 

 Parents feel very positive about their parent educators. Parent educators are considered 

culturally-respectful and competent. 

 The lowest ranking of parent educators had to do with knowledge and efficacy concerning 

children’s challenging behaviors. Helping to plan healthy meals and knowledge of 

developmental delays were also ranked lower than other items, but the mean scores were 

still high (~4.5 of 5.0). 

Program Feedback 

 All programs reported that they achieved what they hoped to with the CPCF-SIF grant. 

 Programs reported increasing capacity to serve new families by hiring new staff, adding 

programs or program components, and expanding services to new communities or families. 

 The CPCF-SIF program added value to the 8 programs by: (1) providing start-up funding to 

add a new program or program component, (2) encouraging integration with the larger 

community by requiring match funding, (3) enhancing services to integrate families better 

into the community, and (4) increasing the emphasis on early literacy. 

 Family service needs and gaps highlighted the ongoing need for: (1) expanding the agerange of the programs, (2) mental health resources, (3) economic resources such as food, 

bills and transportation, (4) more Spanish speaking staff, and (5) specialized information to 

provide to families such as information on prematurity and child development. 

 There is a hunger for reading and literacy, and parent educators helped families build their 

home libraries. Some families didn’t have any books at home prior to the CPCF-SIF study. 

Grant Administration 

 Staff suggested that at the start of a program similar to CPCF-SIF, there should be clear and 

consistent information about the process and requirements. Despite having positive 

feelings about CPCF and the SIF funding mechanism, programs reported challenges to grant 

management, including undue complexity related to invoicing, not knowing who to go to for 

questions, and collecting fairly substantial amounts of data with families. 

 The match funding, while difficult to achieve for some programs, provided a way to 

integrate with community partners and gain momentum, support and enthusiasm for the 

programs.
P a g e | 4

Methods 

This report includes data on all children receiving PAT services at seven participating programs 

over an 18-month period, as well as more extensive data on a subset of children participating in the 

CPCF-SIF research study. The CPCF-SIF study aimed to address questions related to children’s 

developmental functioning and literacy activities between parents and children. Parents of children 

ages 8 to 38 months old at entry to PAT and receiving services through SIF beginning in March 

2013 were invited to participate in the study. (The HIPPY program resourced through SIF 

participated in the study focus groups and interviews, but did not participate in the child and 

parent outcomes study due to resource limitations.) 

The evaluation conducted by the Butler Institute for Families was designed to address three 

primary questions:

 What is the geographic scope and what are the characteristics of families served? 

 What are the early literacy outcomes for children included in the study, including parent 

engagement in early literacy activities, child language development, and communication? 

 What do PAT and HIPPY program coordinators, parent educators, and agency directors 

report about the impact, administration, and sustainability related to participating in the 

Social Innovation Fund through CPCF? 

To assess these questions, three methods were used: (1) tracking of child and parent enrollments 

into PAT programs through administrative data maintained by CPCF, (2) surveys of parents 

receiving services in seven of the eight program sites, and (3) interviews and focus groups with 

staff at the program sites. 

Administrative database. These data, provided by programs to CPCF, include the demographic 

characteristics of all children and parents receiving PAT from the start of SIF through June 2014. 

This includes age, race, ethnicity, language, and gender of children and parents, parents’ household 

income and education status, disability status, and single or two-parent household. Data also 

include free and reduced lunch status of children, entry date to the program, and whether the 

family is served through the SIF specifically. Families who were reported by the PAT programs to 

be enrolled through SIF were invited to participate in the research study. 

Parent surveys. Parents completed surveys at entry to the PAT program and approximately 6 

months later. Surveys were administered by the parent educator. The survey included measures of 

their knowledge and views of child development, home literacy activities, and perceptions of the 

parent educator using an established PAT survey. Parents also completed information about their 

child’s early language skills. This included:

Ages and Stages Questionnaire (ASQ), Communication subscale. Parents responded to six 

questions about their child’s communication, with items varied according to child age. The ASQ 

results in a classification of children that are developmentally on schedule or showing concerns. 

Children with concerns are recommended to be referred to early intervention services for further 

developmental assessment. 
P a g e | 5

MacArthur Communicative Development Inventories (CDI). The CDI-Short Form is a list of 

vocabulary words, sentences, and parent questions designed to measure children’s comprehension 

and production of language. The CDI produces scores that when standardized result in a percentile 

ranking that shows children’s communicative development in comparison with other children the 

same age. 

Staff interviews and focus groups. Interviews and focus groups were conducted by phone with staff 

in each of the eight programs in October and November, 2014. Individual or two-person interviews 

were conducted with Executive Directors, Fiscal Administrators, and Program Coordinators or 

Supervisors. Focus groups were held with parent educators via phone. In total, 18 sessions were 

held including 39 individuals. Questions centered in three areas related to the CPCF-SIF award: (1) 

Impact, (2) Administration, and (3) Sustainability. 

DATA ANALYSIS 

The administrative data were used to describe all children and 

parents receiving PAT in the seven programs from October 2012 

through June 2014. These analyses included descriptive 

proportions, means, and mapping of families receiving services in 

the eight programs across Colorado. 

Children enrolled in the research study were further analyzed for developmental status using the 

ASQ and CDI, and literacy activities reported by parents. Parent reports of their parent educator are 

also included. Analyses used were descriptive (proportions, means) and bivariate using paired ttests and chi-square analysis. 

Qualitative data were analyzed for themes within the three categories of impact, administration, 

and sustainability. Data were also analyzed for overall themes—what were the consistent messages 

expressed and how might programs like SIF use the results to inform future planning? 

PROGRAMS 

The programs that were selected to participate in CPCF-SIF received grant funding to increase 

capacity and provide PAT and HIPPY services relative to the overarching goal of the SIF to increase 

the evidence-base of early childhood literacy programs in Colorado. Eight programs serving families 

in thirteen counties were selected in 2012 to participate. These are shown in Table 1.

 Table 1. Programs selected to participate in the CPCF-SIF, 2012-2014 

Subcontracted Service Partner Counties Program 

Bright Futures for Early Childhood & Families Montrose, Ouray, San Miguel PAT

La Familia/The Family Center Fort Collins Larimer PAT

Starpoint/Developmental Opportunities Fremont PAT

Catholic Charities, Diocese of Pueblo Crowley, Pueblo HIPPY

Family & Intercultural Resource Center Summit PAT

Focus Points Family Resource Center Adams, Denver PAT

Parenting Place Boulder PAT

La Llave Family Resource Center Alamosa, Rio Grande, Saguache PAT
P a g e | 6

 

RESULTS 

PROGRAM AND FAMILY 

CHARACTERISTICS 

This section describes enrollment in PAT across the seven programs overall, as well as enrollment 

through SIF specifically. In total, 1,143 children were served. 

Child demographics. According to 

parent-reported demographic data, 

slightly more than half of PAT children 

across all sites are Hispanic or Latino 

(53%) (Figure 1). Although almost 

two-thirds speak English as their 

primary language (60%), many are 

primarily Spanish-speaking (39%). 

Most PAT children are white (93%), 

and roughly equal proportions of male 

and female children are represented. 

Demographic data indicate that 30% 

of children are eligible for 

free/reduced lunch. However, 

free/reduced status, which was selfreported by parents, was not provided 

for about one-third of children. Also, 

parents may not have reported this 

for children who are not of school age. 

Thus, this measure likely drastically 

underestimates the number of PAT 

children who live close to the poverty 

line. 

The PAT program represented a wide 

range of children by age/grade, from 

prenatal to kindergarten. Most 

children were infants and toddlers. 

The proportion of children by 

age/grade at program entry is shown 

in Figure 2. 

Figure 1. Child gender, race/ethnicity, and primary 

language across the 7 SIF-funded PAT sites 

Gender

(n=1126)

Male, Female, 

51% 49%

Race

(n=1111)

Other, 7%

White, 

93%

Ethnicity

(n=1131)

Not 

Hispanic/ Hispanic/ 

Latino, Latino, 53%

47%

Primary 

Language

(n=1139)

Other, 1%

Spanish, 

39%

English, 

60%
P a g e | 7

Figure 2. Children by age/grade at program entry across the 7 SIF-funded PAT sites 

(n=1119) 

PRENATAL 10%

INFANT 33%

TODDLER 37%

PRESCHOOL 11%

KINDERGARTEN 2%

ECE 7%

0% 5% 10% 15% 20% 25% 30% 35% 40%

Child involvement in PAT. Almost one-third of 

the children in the CPCF PAT sample (31%) 

were enrolled the Mile High United Way 

Social Innovation Fund (SIF) research study 

(Table 2). 

Approximately one-quarter of children in the 

sample (24%) were reported to have been 

served by SIF funding (n=251). 

By PAT site, about one-quarter of children 

were served by the Family Intercultural 

Resource Center (n=256), and approximately 

20 percent each were served by Bright 

Futures (n=204) and Starpoint First Steps 

(n=200). Other sites represented about 10 

percent of children each (n=103-144). 

Table 2. PAT children by enrollment characteristics1

Research enrollment (n = 1143)

 Enrolled in research 31%

 Not enrolled in research 69%

Site (n=1143)

 Bright Futures 18%

 Family Intercultural Resource Center 22%

 Focus Points 11%

 La Familia 13%

 La Llave Family Literacy Center 9%

 Parenting Place 9%

 Starpoint First Steps 18%

Receives SIF Funding (n = 1065)

 Yes 24%

 No 76%

*All new enrollees to the 7 PAT programs (8-38 months old) were 

eligible to participate in the research study regardless of whether they 

were served by SIF funds.

Figure 3 shows the reach of the CPCF PAT program across Colorado by PAT site. Programming 

impacted many parts of the state, including the Denver Metropolitan Area, Northern Central 

Colorado, Southern Central Colorado, the High Country, and the Western Slope. 

The geographical distribution of children served by each 

site typically includes an area in which a large number of 

children were served (100+), as well as a larger area with 

a lower density of children served. For example, for the 

La Familia program, the darker orange area represents 

Fort Collins zip codes, while the lighter orange shows that 

the program extended its reach into other areas of 

Larimer County. 

Darker shades on the map 

represent a greater density of 

children served in a particular 

zip code.
P a g e | 8

Figure 3. 

Number 

of 

children

served
P a g e | 9

Child enrollment by program.

These graphs show the number of 

children by quarter and site who 

entered into the PAT program 

during the period in which the SIF 

project was active. 

For some programs, such as La 

Llave, enrollments varied over time,

while for others, such as La Familia,

they were relatively consistent. 

There were also programs in which 

enrollments seem to have slowed 

over the course of the funding 

period, possibly because the 

program reached capacity. 

Quarterly Program 

Enrollments 

2012-2013 school year 

□ Quarter 1: Sept.-Nov. 2012 

□ Quarter 2: Dec. 2012-Feb. 2013 

□ Quarter 3: March- May 2013 

□ Quarter 4: June-Aug. 2013 

2013-2014 school year 

□ Quarter 1: Sept.-Nov.2013 

□ Quarter 2: Dec. 2013-Feb. 2014 

 
P a g e | 10
P a g e | 11

Parent demographics. 

As shown in Figure 4, most 

parents are white, Hispanic, 

and primarily Englishspeaking. About one-third 

of parents across sites 

speak Spanish as their 

primary language and 

English as a second 

language. 

Overall, demographic 

characteristics of PAT 

parents are similar to those 

of PAT children. 

Level of education was 

available for about one-half 

of PAT parents (Figure 5). 

Of those, one-third had less 

than a high school diploma 

or GED, and roughly onequarter had a high school 

diploma or GED. Just over 

20 percent had at least a 

bachelor’s degree (21%). 

Figure 4. PAT primary parent race/ethnicity and language

Race

(n=941)

Other, 6%

White, 

94%

Ethnicity

(n=991)

Not 

Hispanic/ Hispanic/ 

Latino, Latino, 

48% 52%

Primary Language

(n=972)

Other, 1%

Spanish, 

38% English, 

61%

English as a Second 

Language

(n=997)

Yes, 37%

No, 63%

Figure 5. PAT primary parents by level of education (n=536)1

LESS THAN HS DIPLOMA/GED 33%

HIGH SCHOOL GRADUATE/GED 23%

SOME COLLEGE 15%

ASSOCIATES/VOCATIONAL TRAINING 7%

BACHELORS 16%

MASTERS 5%

0% 5% 10% 15% 20% 25% 30% 35%

1Data on level of education was available for only about 50 percent of PAT parents. 
P a g e | 12

Family-level characteristics 

86% of PAT families had two-parent households (n=848). 

Services for 24% of families were SIF-funded (n= 251). 

Household income was reported by less than one-half of PAT families (n=411). Among the 411 families: 

 31% had incomes less than $20,000 

 39% had incomes between $20,000 and $40,000 

 30% had incomes greater than $40,000. 

Parent educator 

demographics. 

As part of the evaluation of 

the CPCF PAT program, 

demographics were 

collected for a total of 66 

Parent Educators (PEs). 

Like the children and 

parents served in the PAT 

program, PEs are generally 

white and Hispanic or Latino 

(Figure 6). About one-half 

are bilingual (56%), 

speaking both English and 

Spanish equally. 

Nearly all PEs are female 

(94%). 

As shown in Figure 7 (page 

13), many PEs across CPCF 

PAT sites have a bachelor’s 

degree (40%) and 6% have 

a master’s degree. One-half 

have high school to 

associates degree training.

Figure 6. Parent educator gender, race/ethnicity, and language

across the 7 SIF-funded PAT sites

White, 

100%

Race

(n=50)

Hispanic 

or 

Latino, 

67%

Not 

Hispanic 

or Latino, 

33%

Ethnicity

(n=36)

English, 

44%

English & 

Spanish 

Equally, 

56%

Primary Language

(n=50) Male, 6%

Female, 94%

Gender

(n=64)
P a g e | 13

Figure 7. PAT Parent Educators by level of education (n=48) 

6%

40%

12%

21%

17%

4%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

MASTERS 

BACHELORS

ASSOCIATES/VOCATIONAL TRAINING

SOME COLLEGE

HIGH SCHOOL GRADUATE/GED

LESS THAN HS DIPLOMA/GED

In summary, demographic data collected during the evaluation of the SIF project show that a 

diverse group of children and families were reached with PAT services across Colorado. This 

includes many families who were Hispanic or Latino and/or had household incomes below $20,000. 

Parent Educators represented similar demographic backgrounds to PAT families, which may have 

facilitated their ability to engage effectively with parents and children. Most PEs had education or 

training beyond high school, which is likely helpful in working with parents – many of whom did 

not have postsecondary education – to become their child’s first teacher. 
P a g e | 14

RESULTS 

CHILD LITERACY & COMMUNICATION 

The emergence of language is a developmental milestone that parents and families look forward to 

with their children. At the same time, there are many factors that may influence children’s language 

development and ability to communicate with others. Research shows that children who live in 

poverty, for example, have a very large literacy gap compared with other children. Programs such 

as PAT and HIPPY are designed to address children’s development by providing peer-led home 

visiting to parents. During home visits, parent educators and parents engage in reading, play, and 

other activities targeted at skills and education concerning normative development and recognizing 

developmental delays. 

The PAT programs involved in CPCF-SIF (n=7) were instrumental in implementing the research 

study, collecting questionnaires from parents at study intake and over time that included child 

communication measures, the ASQ and CDI. These results are presented next. 

Children’s Ages and Stages Communication Scores. The ASQ Communication subscale includes 6 

items that result in classifying children as above the cut-off (indicating no concerns), near the cutoff (indicating moderate concerns), and below the cut-off (indicating that developmental 

assessment is needed) (Squires et al., 2009). Children with scores below the cut-off show 

communication at a level that is 2 standard deviations below the mean observed among children in 

the general population. Typically, 2.5% of children would score in this range.

Table 3. Percentage of children with ASQ Communication Concern Scores (n=207 total) ^

Infants Early Toddlers Late Toddlers Total

Baseline

(n=42)

Follow-up

(n=9)

Baseline

(n=104)

Follow-up

(n=32)

Baseline

(n=61)

Follow-up

(n=8)

Baseline

(n=207)

Follow-up

(n=49)

0% 0% 5% 12% 8% 25% 5% 12%

^207 children with baseline scores and 49 children with follow-up scores 

As shown in Table 3, there were no infants with ASQ Communication scores in the concern-range 

at study enrollment or follow-up. Among early toddlers at baseline (age 1 to 2), 5% had ASQ scores 

in the concern range, and 12% had ASQ concern scores at follow-up. Among late toddlers at 

baseline (age 2 to 3), 8% had ASQ concern scores and 25% had ASQ concern scores at follow-up. 

Among all children, 5% had ASQ concern scores at baseline and 12% had ASQ concern scores at 

follow-up. 

This shows that children enrolled in the CPCF-SIF study have higher need for developmental 

assessment related to communication relative to children in the general population. Compared to 

2.5% of the general population, CPCF-SIF children have baseline concern-rates that are double the 

rate expected in the general population. At follow-up, CPCF-SIF children have concern rates that are 

4 to 5 times higher than rates expected among other children. This may be because the number of 

children with ASQ follow-up measures was small (n=49) and could possibly reflect parents that 

were more motivated to complete the measure due to having concerns. Regardless, results clearly 

depict above-average communication challenges among the toddlers, particularly, in this study. 
P a g e | 15

What factors predict ASQ scores? We examined child and family factors that may be associated 

with children’s language development using the ASQ measure at baseline. Factors that may relate to 

children’s language development are age, gender, primary language and ethnicity, family type 

(single-parent vs. two-parent) and household income (< $20,000 vs $20,001 and above). We also 

viewed the relationship between ASQ scores and the frequency of literacy activities in the home 

reported by parents. We included these factors in a multivariate, predictive analysis of child 

communication classification at baseline as “concerns” or “no concerns.”

Results showed that gender and the frequency of literacy activities in the home were 

significant predictors of ASQ scores. Specifically, boys were 3.8 times more likely to have scores 

below or near the cutoff at baseline compared with girls. Children whose parents reported fewer 

literacy activities in the home were 3.3 times more likely to have ASQ concern scores compared 

with children whose parents reported more literacy activities. These results are shown in Figures 8 

and 9. There were no other child- and family factors that predicted ASQ concern scores. See 

Appendix 1 for the full model. 

Figure 8. ASQ Communication concern-rates among boys (n=99) compared with girls (n=84) 

100%

80%

60%

40%

24%

20% 11%

0%

Boys Girls

% 

of

children 

NEED FOR DEVELOPMENTAL 

ASSESSMENT 

Boys were 4x more 

likely to have ASQ 

communication concern 

scores compared with 

girls at study 

enrollment. 

Every day 5

Every other 3.8

day 4

3.2

Once or twice 3

a week 

Less than 2

once a week 

Never 1

Concerns No concerns

Figure 9. Mean frequency of home literacy activities 

Mean 

Score 

Statistical modeling showed...

A child whose parent 

reported lower levels of home 

literacy activities was 3.3 

times more likely to have ASQ 

concern scores compared to a 

child whose parent reported 

more literacy activities. 
P a g e | 16

MacArthur-Bates Communicative Development Inventory (CDI) 

Language comprehension and production. CDI short forms were used in this study (Fenson et al., 

2007). The short-form for infants includes an 89-word checklist for vocabulary comprehension 

(“words understood”) and production (“words produced”). The toddler short-form includes a 100-

word vocabulary production checklist and a question about word combinations. By tallying the 

number of words, children are given percentile rankings, or the ranking of CPCF-SIF study children 

in comparison with children in the general population. These results are presented in Table 4.

Table 4. CDI percentile rankings of CPCF-SIF study children 

25th percentile 

and below 

26th-50th

percentile 

51st-75th

percentile 

Above 75th

percentile

N % N % N % N %

Infants

(8-15.99 months)

Words Understood 20 25.3 17 21.5 17 21.5 25 31.6

Words Produced 21 26.6 24 30.4 13 16.5 21 26.6

Early Toddlers

(16-30 months)

Words Produced 58 42.0 35 25.4 27 19.6 18 13.0

Combining Words 1 0.7 11 8.0 37 26.8 89 64.5

NOTE: only 21 “later toddlers” had a baseline CDI

Percentiles were obtained using the CDI Scoring Program (Marchman, 2013). 

Among infants, 43% fall in an average-range of words understood (26th-75th percentiles), and onethird are above-average (above 75th percentile; 32%). One-quarter are at the 25th percentile or 

lower at intake to the CPCF-SIF study. These scores are comparable to scores expected among other 

infants. 

Many infants in the CPCF-SIF study are also average in their word production, with 46.9% in the 

26th to 75th percentile ranges. About one-quarter are at or below the 25th percentile (26.6%), 

ranking lower than other infants, and another one-quarter are above the 75th percentile (26.6%), 

ranking higher than other infants. 

While infants are roughly equivalent to other infants in the general population regarding word 

production, early toddlers in the CPCF-SIF study are below-average in word production. Among 

138 early toddlers with intake CDI scores, 42% were at or below the 25th percentile in word 

production. Just 13% were above the 75th percentile in word production, and 45% were average 

(26th to 75th percentiles). 

Early toddlers rated more highly in combining words—64% of early toddlers were above the 75th

percentile at study intake and another 35% were average, in the 26th to 75th percentile. 

42% of early toddlers in the 

CPCF-SIF study are below average 

in word production, a rate that is 

68% higher than other children. 

Most infants in the study are “on 

course” or average in word 

comprehension and production.
P a g e | 17

CDI Norms for combining words 

(% in general population expected to 

combine words by age and gender)*

Age (months) Girls Boys

16 35.1 21.6

17 50.0 32.6

18 65.9 44.1

19 78.2 53.5

20 70.0 68.4

21 72.5 72.7

22 86.0 72.5

23 84.6 96.1

24 85.9 85.7

25 98.3 87.5

26 95.7 88.7

27 96.3 91.5

28 97.6 95.3

29 97.4 100.0

30 100.0 100.0

*Fenson et al. 2007

Combining words. Combining words is also an 

important developmental milestone, and among 

girls, nearly all (95%) should be combining words by

age 2 to 21⁄2 years (24-30 months). Among boys, 

about 87% or more should be combining words at 

age 2 to 21⁄2 years. Table 5 presents the proportion 

of boys and girls combining words in the CPCF-SIF 

study at intake.

 

Table 5. Combining words among study children

Girls

(n=61)

Boys

(n=64)

16-20 months (n=28) 62.6 58.3

20.1-24 months (n=37) 82.3 70.0

24.1-30 (n=60) 100.0 93.8

In the CPCF-SIF study, results showed that 100% of girls were combining words between ages 2 to 

21⁄2, and 93.8% of boys were combining words at the same age. This shows that PAT children 

participating in the CPCF-SIF study were comparable to other children in terms of the level of 

combining words measured by the CDI.

Prior to age 2, there should be a linear increase in 

the proportion of children combining words, from 

roughly 35% of girls and 22% of boys at age 16 

months and progressing to 86% of both boys and 

girls at age 24 months. Among CPCF-SIF children, 

two-thirds of girls, age 16-20 months, were 

combining words, which is similar to the CDI norm 

at 18 months. For boys, age 16-20 months, 58% were combining words, which is close to the 19-

month norm. For children age 20 to 24 months old, CPCF-SIF study children were similar to CDI 

norms at 22 months for both genders. 

Overall, CPCF-SIF study children appear to be on course in language development measured by the 

CDI vocabulary checklist and combining words item. The most notable gap is among early toddlers, 

wherein the proportion of children with low scores is 42%. This is nearly 70% higher than the 

expected rate of 25% of children. 

Combining words is defined as a 

parent response that their child 

“sometimes” or “often” combines 

words, such as “nother cookie” and 

“doggie bite”.
P a g e | 18

were high, with average scores of 4.0 or 

greater, indicating moderate-to-high ratings. 

It is notable that scores were slightly higher for 

items about supporting their child’s 

development and learning, versus recognizing 

potential developmental issues. 

Although some scores went up or down 

between time points, changes were not 

statistically significant.1 However, parents’ 

confidence in knowing how to find 

resources showed a marginally significant 

increase.2 Thus, it appears that parents are 

slightly more confident in their ability to find 

resources after taking part in PAT. 

RESULTS 

PARENT SELF-RATED KNOWLEDGE 

AND SKILLS 

In the PAT Parent Survey, parents were asked to rate their knowledge and skills related to child 

development, parenting practices, and child health. Results are summarized in the sections below. 

Parent support of child’s emerging 

development. The PAT program aims to help 

parents understand their child’s development, 

including strengths and possible delays. 

Six questions assessed parents’ level of 

confidence in their knowledge of child 

development, including their understanding of 

developmental milestones, their ability to 

recognize possible developmental issues, and 

how to support their child’s learning. 

Items were rated on a scale of 1= “no 

confidence at all” to 5 = “a lot of confidence” 

Results are shown in Figure 10. 

Parent ratings of their confidence related to 

understanding their child’s development

Results are reported for families who 

took the survey when they enrolled in 

the study (Time 1), and approximately 

six months later (Time 2). 

 

 

INTERPRETING THESE RESULTS 

The small amount of change over time may be due to: 

 The high ratings at Time 1 – there is no room for improvement 

 The scale or the method of assessment (parent report) 

 A small group with post assessments 

 Insufficient amount of time between assessments to be able to 

detect change. 

Figure 10. Parent self-ratings of skills to support their child’s development 

(n=75) 

5

4 4.5 4.3 4.2 4.3 4.3 4.3 4.3 4.1 4.1 4.2 4.0 4.2

3

2

1

Supporting Interacting Age Where to find Recognizing Recognizing

your with your child appropriate resources potential possible

child's to support developmental health developmental

learning development milestones problems delays

Time 1 Time 2 A lot of 

confidence 

No 

confidence 

at all 

1According to paired samples t-tests and Wilcoxon signed rank tests for analysis of nonparametric paired samples. 

2 p = .05
P a g e | 19

Knowledge of child development. 

The survey also had six true/false questions 

assessing parents’ knowledge of child 

development concepts. 

Results are shown in Figure 11. 

Parents generally answered questions 1-4 

correctly, indicating that they understand 

the nature of how very young children learn 

and the influence of parent engagement. 

Because these items were answered 

correctly so consistently, there was little – if 

any – room for improvement over time. 

There were no statistically significant 

differences in scores over time for Items 1-4. 

For Items 5 and 6, lower proportions of 

parents (60%-90%) answered correctly, 

indicating that some were unsure about the 

impact of interacting with their child on 

his/her development. Scores for Item 6 

showed a statistically significant decrease

over time.1 However, because the 

statements were “false,” it is possible that 

the negative wording of these questions was 

confusing. 

INTERPRETING THESE RESULTS 

Consistent scores over time may be due to: 

 High ratings at Time 1 – no room for 

improvement 

 Parent self-report 

 Small number of responses to detect changes 

 Insufficient duration between assessments. 

 

 Survey Items on knowledge of child development 

1. Children are learning from moment they are born. 

2. Parents' emotional closeness with their child can strongly 

influence their child's development. 

3. The things a child experiences before the age of three will 

greatly influence his/her ability to do well in school. 

4. If a child does not receive appropriate stimulation... his/her 

brain will develop as well as the brain of a baby who does 

receive these types of stimulation. 

5. Every child is born with certain level of intelligence, which 

cannot be either increased or decreased by how parents 

interact with him/her. 

6. The more stimulation a child receives by holding and talking 

to them, the more you spoil them. 

Results are for families who took the survey when they enrolled in 

the study (Time 1), and about 6 months later (Time 2). 

 

Figure 11. Parent knowledge of child 

 development (% correct; n=74)

100% 1. 96%

100% 2. 97%

97% 3. 97%

96% 4. 97%

5. 63%

65%

91% 6. 76%

0% 20% 40% 60% 80% 100%

Time 1 Time 2

Parenting practices. To assess how confident parents feel in their knowledge and ability to use 

appropriate parenting practices, the survey included four questions about “good parenting practices” and 

“positive discipline techniques.” Results are shown in Figures 12 and 13. 

Figure 12. Parenting practices (n=71-74) 

No confidence A lot of 

at all confidence 

Know good 4.4

parenting practices 4.3

Use good parenting 4.4

practices 4.4

1 2 3 4 5

Time 1 Time 2

Figure 13. Positive discipline techniques 

(n=73-74)

No confidence A lot of 

at all confidence 

Know positive 4.1

discipline techniques 4.3

Use positive discipline 4.2

techniques 4.4

1 2 3 4 5

Time 1 Time 2

1Item 6 was statistically significant according to a Wilcoxon signed rank tests for analysis of nonparametric paired samples (p = .012). 
P a g e | 20

As shown, parents are generally confident in their parenting skills, with mean scores greater than 4.0. 

There were no significant changes in scores over time, but there were slight increases in scores for the use 

of positive discipline techniques. 

Child health. Parents were also asked to rate 

their knowledge of six key issues related to a 

child’s health and well-being. These items 

were rated of a scale of 1 = Not at all 

knowledgeable to 5 = Very knowledgeable. 

Figure 14 shows these results. 

 

INTERPRETING THESE RESULTS 

The small amount of change over time may be due to: 

 The high ratings at Time 1 – there is no room for improvement 

 The scale or the method of assessment (parent report) 

 A small group with post assessments 

 Insufficient amount of time between assessments to be able to 

detect change. 

Figure 14. Parent-reported 

knowledge of child health at enrollment in the PAT study and 6 months later 

(n=91) 

5

4 4.7 4.7 4.7 4.6 4.6 4.5 4.6 4.3 4.4 4.5 4.4 4.4

3

2

1

Importance of A healthy diet How to Importance of Health risks of How to

well-child for a child prepare a child being childhood prevent

visits healthy foods physically obesity childhood

active obesity

Time 1 Time 2

Very 

Knowledgeable 

Not at all 

knowledgeable 

Overall, parent ratings of their knowledge and skills were positive and did not change significantly over 

time. There are several possible explanations for the consistency of scores over time in these self-reported 

scores: 

 With favorable scores at Time 1, there was little room for improvement. 

 With scores clustering around 4.5 on a 5-point scale, perhaps respondents were reluctant to assign 

themselves the highest rating. 

 A relatively small number of respondents (approximately 70) completed the survey at Time 1 and

Time 2, and significant increases over time might be detected with a larger sample. 

 Some families completed Time 1 assessment after they had been in the program for several months. 

Parents’ highest scores were for understanding the importance of well-child visits and being physically 

active, with mean scores of 4.6 or greater. Scores were slightly lower for items about parents’ knowledge 

of how to encourage a healthy lifestyle for their child, such as by preparing healthy foods and preventing 

childhood obesity. There were no statistically significant differences in parent-reported knowledge of 

child health over time. 
P a g e | 21

RESULTS 

HOME LITERACY ACTIVITIES

Families reported engaging in several literacy 

activities more frequently over time. 

Parents were asked about the frequency with which 

they or others in the household engage in literacy 

activities with the child in a typical week. 

Results are shown in Figure 15. On average, 

parents or caregivers sing songs, tell stories, read 

board books, and have conversations with their 

child about every other day. 

Between Time 1 and Time 2, there were 

statistically significant increases in scores for items 

on talking about books and stories without words, 

as well as writing/talking about the child’s name 

and looking at/talking about shapes.1

It is important to note that some parents reported 

that children were “too young” for certain activities 

Time 1. Thus, gains over time may be due to a child 

“aging in” to the activities. However, even among 

only children who were “old enough” at both 

points (n=39), there was a significant increase over 

time for talking about books (p < .01). 

Figure 15. Frequency of literacy activities in the home (n=94 -99)

5

4

3

2

1

Time 1 Time 2 Every day 

Significant

Every other Increase

day 

Once or twice 

a week 

Less than 

once a week 

Never 

** p < .01 * p < .05 

1According to paired samples t-tests and Wilcoxon signed rank tests for analysis of nonparametric paired samples. 
P a g e | 22

RESULTS 

INTERACTIVE READING TECHNIQUES 

Parents were asked about the techniques they use when reading with their child and whether they 

engage interactive reading techniques, such as pointing out letters and words, talking about the story 

with the child, letting the child turn the pages. 

Results are shown in Figure 16. Overall, parents were “in the middle” of the scale in terms of how 

often they use the interactive techniques. There were statistically significant increases over time in 

the frequency of guessing what will happen next, as well as encouraging the child to read along and 

repeat simple rhymes or phrases.1 This was among the full sample of children (n=95-99). However, 

when the analyses were limited to children whose parents felt they were old enough for the 

activities at both time points, there were no statistically significant differences. This indicates that 

increases were likely due to children “aging into” the activities over time. 

Figure 16. Frequency of interactive reading techniques in the home (n=95-99)

5.0

4.0

3.0

2.0

1.0

Time 1 Time 2

Frequently 

 

Significant 

 Increase

Never 

 

** p < .01 * p < .05 

1According to paired samples t-tests and Wilcoxon signed rank tests for analysis of nonparametric paired samples. 
P a g e | 23

In the survey, parents also provided feedback on their parent educator (PE), including his/her 

knowledge and effectiveness at helping families. An overview of their responses is provided below. 

RESULTS 

PARENT PERCEPTIONS OF PARENT 

EDUCATORS 

Parent educator knowledge 

(At Time 1) 

 OVERALL, PARENTS BELIEVE PEs ARE 

KNOWLEDGEABLE ABOUT CHILD 

DEVELOPMENT AND PARENTING TOPICS. 

 SCORES WERE CONSISTENT OVER TIME. 

 HIGHEST RATINGS: KNOWLEDGE OF 

WELL-CHILD VISITS (4.70) & HEALTHY 

DIET/FOODS(4.60) 

 LOWEST RATING: KNOWLEDGE OF 

DEVELOPMENTAL DELAYS (4.44) AND 

HOW TO DEAL WITH A CHILD’S 

CHALLENGING BEHAVIORS (4.39) 

Parent educator effectiveness 

(At Time 1) 

 OVERALL, PARENTS BELIEVE PEs ARE 

EFFECTIVE AT PROVIDING SUPPORT TO 

THEIR FAMILY. 

 SCORES WERE CONSISTENT OVER TIME. 

 HIGHEST RATING: DOING A GOOD JOB 

OVERALL (4.72) AND SUGGESTING 

RESOURCES (4.68) 

 LOWEST RATING: HELPING DEAL WITH 

CHALLENGING BEHAVIORS IN YOUR 

CHILD (4.59) AND HELPING PLAN 

HEALTHY MEALS FOR CHILD (4.55) 

As shown in Figure 17, feedback on PEs’ respect for the family’s culture and beliefs and 

responsiveness to parents’ questions was very positive, with mean scores close to 5.0 – the highest 

possible rating. 

Figure 17. PE respect and responsiveness 

4.8

4.9

4.8

4.9

1.0 2.0 3.0 4.0 5.0 6.0

How responsive is your parent educator to

your questions about parenting?

How respectful is your parent educator of

your culture and beliefs about parenting?

Time 1 Time 2

Not at 

all

 Very
P a g e | 24

RESULTS 

FEEDBACK FROM STAFF IN 

INTERVIEWS AND FOCUS GROUPS 

Qualitative phone interviews were conducted with staff at each organization. There were 18 total 

sessions including 39 individuals. Of the 39 individuals, 8 were program coordinators, 11 were 

executive directors or fiscal management staff, and 20 were Parent Educators. This section 

summarizes the feedback from staff, using excerpts from interview notes.1

Table 6. Participants in interviews and focus group participants 

Role Number of 

participants

Executive Director and Fiscal 11

Program Coordinator 8

Parent Educators 20

Total 39

The sessions included 12 to 15 questions 

centering on successes and challenges relative 

to participating in CPCF-SIF, and needs of the 

program and families. Questions were designed 

to address impact, administration, and 

sustainability. While questions varied somewhat 

according to participants’ roles, several 

questions were asked of all participants (see 

Right). 

Parent educators were asked specific questions 

about program services—benefits to families, 

family needs and service components related to 

SIF. All management staff were asked to 

comment on sustainability, and how a program 

like SIF could further assist them in their work. 

Qualitative data were analyzed for themes and 

grouped into the three categories of impact, 

administration, and sustainability. Responses 

were also analyzed for the overall story—what 

were the consistent messages expressed and 

how might programs like SIF use the 

information for program improvement? 

Some questions asked of all participants: 

1. Did you achieve what you hoped to achieve with 

this grant? (How so or why not)? 

2. How did implementation processes for your 

program change as a result of participation in 

this? (e.g. changes in staffing, professional 

development or coaching, program structure)? 

3. What overall impact did the SIF have? What did 

SIF allow you to do? Was there value add? 

4. What was your greatest challenge? What 

worked well? What would you change if you had 

a “Do Over” button?

5. Think about an amazing story – an aspect of the 

work that stands out to you in a positive way. 

This could be about families, staff or impact in 

the community. Please share this amazing story. 

1Although is likely that interview notes capture many comments from interviewees word-for-word, it cannot be 

guaranteed that excerpts from interview notes are verbatim quotations. 
P a g e | 25

Impact. All programs reported that they achieved what they hoped to achieve with the grant. 

Several discussed how the grant increased capacity and a few discussed how they were able to 

benefit families through the services offered and further integrate their organization into the 

community. There were two programs that noted there were not specific impacts of the CPCF-SIF 

funding, but rather the funding allowed them to continue what they were already doing through 

supporting programs already in place. 

Participants reported that the primary impact was increasing capacity—to serve families, to 

provide a new aspect of programming, or to continue their existing program with greater reach. 

Increased capacity. Most organizations reported that capacity increased from receiving the SIF 

grant. They were able to hire and train new staff, increase the number of families served, expand 

their service area, and add to the services they offer. Among the programs, 5 reported using SIF 

funding for new staff, 4 reported using SIF funding primarily for additional or new services, and 4 

used SIF funding primarily to expand their reach to a new county or population not already served. 

New Staff. Five organizations reported that they hired and trained new Parent Educators in order to 

serve more families. Other hired staff also included coordinators, data management staff, assistants, 

and even volunteers. 

Increasing the number of families served. Four programs specifically noted that they increased the 

number of families served, either by expanding to a new county or area, or by providing services in 

an underserved population. As mentioned, hiring new staff helped increase the number of families 

served. Other organizations were able to increase the number of families that current Parent 

Educators visited. 

Staff members working in rural areas interviewed said that they were able to increase the areas 

that their program reached given that large parts of surrounding areas have not qualified for other 

types of funding: It allowed us to expand to an area that wasn’t allowed to be served. – Program 

Coordinator. 

A couple of staff members mentioned how the Latino community was able to be served with the 

grant, Integration is one of my goals but in our community, it is a challenge; the Latino community 

here is not well integrated they are seen as the working class; I think what you are doing is making an 

impact on families and it is achieving its objectives; I think it’s amazing; we have real engagement 

with our families and parents have better engagement and through the opportunities that we tell 

them about. - Parent Educator/Coordinator

Adding programs or services. Four programs specifically discussed new services added related to the 

CPCF-SIF grant. The following list details some of those services. 

 HIPPY program 

 Bilingual family nights 

 The additional information provided during home 

visits. Home visiting offers passports, which are not like 

real passports. They are a book that provides 

information such as healthcare and other resources that 

we could offer them. – Parent Educator

Most definitely, without the SIF 

program we wouldn’t have been 

able to start the HIPPY program 

in Pueblo. Without those funds 

we would not have brought that

program here. – Coordinator
P a g e | 26

Value-add. Respondents were asked whether there was “value add” to participating in the CPCFSIF program. Responses varied, but most programs mentioned the value of start-up funds for 

HIPPY, PAT, or an aspect of programming. Value-add also related to integration, literacy, and 

increasing capacity.

Table 7. What was the “value-add” to participating in CPCF-SIF? 

Response Example

Start-up of either HIPPY 

or PAT 

I mean I keep going back to the initial getting started. I really do 

believe if we did not have this money we wouldn’t have had the 

funds to start the program when we needed to start it. And being 

able to sustain it the second year. – Program Coordinator

It created a consistency in our bilingual story time event that was 

not there before. We had the same parent educators working that 

event each month and it created routines and boosted the quality of 

the event. – Program Coordinator

The organization 

integrating more heavily 

into the community

Organizations reported that the match funding requirements 

helped with how they were able to find support in their 

communities and build their networks.

Since starting HIPPY, it integrated our agency more into the 

business community because the business community really bought 

into the HIPPY model. – Executive Staff

Helping families to 

integrate better with the 

community, for social 

support or resources

Families were able to build their networks of community 

resources by having increased support from Parent Educators. 

Several staff stated that they would accompany parents to 

hospitals, schools, and food banks, among other resources. 

In rural areas, staff mentioned how the home visits to families 

provided relief for isolation and encouraged families to connect 

with the community. 

Increased capacity The increase in capacity of additional staff really helped the 

organizations by easing the workload of other staff while being 

able to increase the services offered. 

The importance of 

financial support

Many organizations mentioned how much the SIF’s financial 

support enabled program continuation, sustainability, or 

provision of services. 

A larger focus on literacy The early literacy emphasis in CPCF-SIF that provided books, 

curriculum, and additional literacy services for families was a 

“value add,” such as the bilingual family night: 

[The bilingual family night] helped with early literacy, getting books 

to the families. It helped build libraries at home that they didn’t have 

before. – Parent Educator 

It reminded us all just by having two years thinking of literacy... it 

really helped to drill that into our Educator’s minds. -Executive 

Staff/Coordinator
P a g e | 27

Family service needs met and gaps. The PAT and HIPPY programs will likely continue as longstanding evidence-based models of early childhood home visiting. However, are there gaps in what 

PAT and HIPPY provide? Parent educators were asked about specific family needs that are met 

through PAT and HIPPY and the gaps that still remain. The following are the main themes reported 

by parent educators. 

Family Needs Met 

 School readiness and preparation 

 Literacy building 

 School advocacy 

 Child development 

 Connecting families to community 

resources 

Gaps Remaining

 Expanding the age range of PAT or 

HIPPY overall or a couple of years into 

the school system

 Economic support – bills, food, 

transportation 

 Mental health resources

 Additional information – limited 

supplemental information, premature 

children, child development

 More Spanish speaking staff

Family service needs met. Parent Educators spoke very highly of the PAT and HIPPY programs and 

mentioned how beneficial it was for families to receive home visits. They were able to prepare 

families for school, increase preschool enrollment, and educate parents about early childhood 

literacy and development. The home visits also provide a way to connect families to local resources, 

and intervene when families were having health or developmental problems. This Parent Educator 

reports:

Every visit I go, I bring a basin of books and every kid gets 

to get a book and keep it- they love it and that way I am 

always sure that the home has books in it- it’s almost like 

Santa, I think that there is this hunger for reading and 

literacy and they realize that this is something important 

to them- we build a love for reading. – Parent Educator 

Gaps remaining. However, there were many service gaps 

remaining for families. A Parent Educator mentioned: There are a lot of [gaps]; they need help with 

their costs. But we don’t know if this is a PAT objective because we are working with such low income 

families [Translated].” 

The remaining needs were mainly around additional services and information such as 

prematurity, as well as adjusting current services to families with more Spanish speaking staff

and expanding the age-range of services to older children, such as several years into school. 
P a g e | 28

Grant administration. The overarching finding reported by programs regarding the administration 

of the CPCF-SIF grant was that there was inconsistency—some programs felt that the 

administration process was smooth and clear, and others thought it was exactly the opposite. 

Several financial management staff mentioned the difficulty of invoicing and others mentioned how 

easy it was because of the reporting format. One program coordinator reported that, No, it was easy. 

It was actually really easy to coordinate this program with the funds, while another stated, I think 

that there were a lot of people involved in the chain of communication. I felt confused as to who I 

should be asking my question to.

Some Parent Educators were frustrated with the data collection process while others were happy 

with the information that they were gathering. Staff from the same organizations even differed on 

their views of the difficulties or ease of grant administration. Overall, however, staff expressed 

positive relationships with CPCF, and the responsiveness of CPCF staff, even when there was 

inconsistency in the process or confusion about requirements. 

Executive Directors and Fiscal Management Staff. Many 

executive and fiscal staff stated that the financial side 

of the grant administration was very challenging. 

Administrative staff mentioned that the invoicing and 

reporting for the grant was difficult and confusing. 

Several organizations mentioned the difficulty of 

matching and the matching process. Others reported 

that there was a lag in communication and confusion 

around the requirements of the grant. 

Two programs mentioned that the grant was fairly easy to manage and that they appreciated the 

reporting format. 

Program Coordinators. As mentioned before, there was not really a consensus among coordinators 

about the ease or difficulty of administration. However, most coordinators did state that the 

program implementation was simple while the grant requirements were confusing. 

Parent Educators. Parent educators expressed some concern that the data collection took away from 

the amount of time available to do the substantive work of home visits. In some communities, 

parent educators added an extra home visit to accomplish getting measures from families, and in 

other communities, a home visit would be mostly replaced by helping families complete the tools. 

Parent educators discussed the time lags in receiving the evaluation surveys, and difficulty when 

surveys were sent in the wrong language or age-range. Some Parent Educators mentioned how the 

additional questions—specifically, income—created mistrust between them and the families, and 

that some families opted to leave the program because of evaluation requirements. 

These were the major points discussed by parent educators, which, importantly, had to do with 

evaluation and research rather than administration related specifically to CPCF. 

Positively, some staff discussed the utility of the CDI—the list of vocabulary words really helped 

parents recognize gaps in words their child was saying, and to understand words that might be 

expected given the age of their child. 

The following table summarizes the findings related to grant administration. 

From an economics of scale perspective, 

the monthly invoices for SIF were about 

$3,000 and it took more hours each 

month to even process the invoices for 

that than for other grants for $200-

300,000 reimbursement.

– Financial Management Staff
P a g e | 29

Table 8. Successes and challenges in grant administration reported by participants

Evaluation learning. Because the Social Innovation Fund aims to increase the evidence-base of early 

literacy programs, there were research requirements of programs. Participants were asked about 

any changes in evaluation capacity resulting from CPCF-SIF, and learnings regarding the evaluation. 

While many programs expressed a minimal amount of evaluation capacity-building related to the 

grant, some expressed becoming more data-driven throughout the process. Most programs looked 

forward to receiving data back from the surveys they administered as a learning opportunity.

 

Successes Challenges Grant Matching

The idea of the match worked well, it drove 

up the enthusiasm in the community; it 

helped even drive some local community 

money; I think people liked that part and I 

haven’t really had that come along often. –

Fiscal Management

The match piece was a challenge – we were not 

allowed to use the other federal grant money. –

Executive Staff

Requirements

From the financial standpoint, everything 

was straight forward, most was for salary 

for programs. – Fiscal Management Staff

By following SIF requirements, we were 

able to take a look at figuring out where our 

parent educators were in regard to 

education and experience and where we 

wanted them to be. – Financial Management

The whole thing was a little convoluted and created 

an administration burden that wasn’t worth the funds 

we were getting – Executive Staff/Coordinator

We considered cancelling and withdrawing 

sometimes, we felt far too much effort between what 

we were told and the money we were getting. –

Coordinator

Communication

Like I mentioned for me it was really helpful 

when the coordinator in Denver began to 

interact with the educators on an individual 

level. I was still getting notifications but not 

having to do paperwork at a 150 mile 

radius was really helpful. I was happy for 

him to take that over and that was really, 

really helpful for all of the staff and for me, 

it saved a lot of headaches. – Executive 

Staff/Coordinator

I think that there were a lot of people involved in the 

chain of communication (and sometimes) I felt 

confused as to who I should be asking my question 

to. Also, lot of good clear information presented in 

beginning when we started SIF would have been good.

- Coordinator

Data Collection

I think it helps when we do a pre and post 

survey because it shows us as a home visitor 

what we can work on or where we’re at. It’s 

great to see where they end up but it’s good 

to see how they are starting off as. – Parent 

Educator

There seemed to be no parameters for data collection. 

It was a frustrating grant. – Executive Staff

It was [a] requirement and sometimes challenging. 

We use other assessment tools for other things and 

sometimes we don’t have enough time to do the 

activity and curriculum and do all the paperwork with 

parents. So, we can’t be actively engaged in the

activity as well. – Parent Educator

As we’ve matured as a program we’ve become more data driven. As a culture, we’re moving towards that. 

Making sure that we’re more accurate and that we have more people doing that. – Program Coordinator
P a g e | 30

Remaining program needs: What would you change if you had a “Do-Over button” 

and how could a program like SIF further assist you? 

More communication surrounding grant requirements and expected outcomes: 

 Setting down parameters before we start. I don’t feel like anyone was trying to make it 

harder than it was, but understanding what we needed to do from the start would have been 

better. – Financial Management Staff 

 I guess understanding exactly what SIF’s purpose was and knowing exactly what they were 

trying to achieve from the beginning. – Executive Staff 

Additional data support: 

 They can help us lobby with VisiTracker to make it more streamlined and make reports more 

accessible. Just funny little things like how many kiddos are enrolled in your program...we can 

only see families, things like that. Or how to get more data to get funding. – Coordinator 

 Receiving any of the data back from what we collected so we can see what the results and 

efforts were. – Coordinator

 I would say if something like this continues where I feel like it’s supposed to be around, having 

programs be more data driven, being able to do more. I think if there was a really great 

plan in place before distributing the funds or receiving them, it would be able to have a lot 

more impact for data... for funding purposes or having more of an impact in the community. If 

something like this would continue if they were having more of a streamlined purpose. –

Executive Staff

Additional funding or capacity building: 

 The funding itself is really great in the way it was designed for literacy. Because that’s what 

we’re about, school readiness and early literacy. Just being able to build that into our program. 

We have so many children under the age of 5 from low income high risk families. Just the more 

people that we could serve if we could get more funds. You always hear about the word gap 

and are you really making a difference if you’re serving 180 children out of tens of 

thousands? – Coordinator 

Grant requirements: 

 The match piece – I am fine giving reports with the funds, but the match piece was like “this is 

not allowed, this is allowed. For me, there was a learning curve. – Financial Management 

 The invoicing. Everything really did go well and I mean it’s a program being administered by 

other organizations that we weren’t able to administer here. The SIF requirements are similar 

to other programs following fidelity. – Executive Staff 

No changes: 

 You know I think whenever I called them, we mostly emailed back and forth, they’ve always 

been very responsive. – Financial Management Staff

 I don’t think there was any other way they could have assisted us – it was pretty good the way 

it was. They provided a lot of technical assistance and, as things changed at the federal level, 

[...], the accountant that was with CPCF. She answered all of our questions. They could not 
P a g e | 31

have done any better – she provided clear, up to date answers and was available on the 

finance piece. – Executive Staff 

Sustainability. Most programs were able to sustain the changes that were funded with the SIF grant 

by receiving additional funding. Some mentioned that the SIF grant was only a small part of their 

funding so they were easily able to transition. These changes include the additional program staff, 

additional services such as the bilingual family night, and families/areas served. Many staff 

members also mentioned that their program has continued to expand. 

We still have that parent educator, so it allowed us to have a couple years to figure out what 

we were going to do to keep on a parent educator; and we were able to figure it out. The grant 

was helpful, it gave us some time to figure out our future and to get someone in the door and 

trained to become a parent educator. It really helped. – Financial Management Staff 

A small minority of organizations in rural areas that rely on local funding are having trouble finding 

funding from other sources. Some mentioned that there was not a large corporate response in their 

area or that they were ineligible for certain funding because of their region. 

We’re having a hard time. There was the SIF grant and the new director is writing all these 

grants for the first time, and some other grant didn’t go through that we qualified for but we 

were ineligible because of our region. We’re having a hard time with funding right now, we’re 

having to look for local resources. We’re sure hoping to sustain what we have. – Coordinator 
P a g e | 32

References 

References 

Fenson, L., Marchman, V.A., Thal, D. J., Dale, P.S., Reznick, J. S., & Bates, E. (2007). Mac-Arthur 

Bates communication development inventories: User’s Guide and Technical Manual (2nd ed.). 

Baltimore, Maryland: Paul H. Brooks Publishing Co., Inc. 

Marchman, Virginia. (2013). Scoring Program for the MacArthur-Bates Communicative 

Development Inventories. (2013) [CDI Utilities Scoring Program]. Retrieved from http://mbcdi.stanford.edu/scoringdb_p.htm. 

Squires, J. , Bricker, D, Twombly, E., Nickel, R., Clifford, J., Murphy, K., Hoselton, R., Potter, L., 

Mounts, L., & Farrell, J. (2009). Ages and stages questionnaire: Third edition. Baltimore, 

Maryland: Paul H. Brooks Publishing Co., Inc. 

U.S. Census Bureau. (2011). Education level, adults 25 and older. Retrieved from http://www.census.gov 
P a g e | 33

Appendix 1. Logistic Regression Model Results 

As shown below, the model predicting ASQ results for PAT children (concern/no concern) was 

statistically significant, χ2 (1, N=117), = 19.52, p = .01. This means that the child- and family-related 

factors, as a set, significantly distinguished between children with an ASQ score above or close to/below 

the clinical cutoff. Model fit was acceptable, χ2 (8) = 11.49 p = .17. Two predictors significantly predicted 

ASQ scores, child gender χ2 (1, N=117) = 5.1, p = .01 and frequency of literacy activities χ2 (1, N=117 = 

10.2 p < .01. 

Predictor β SE β Wald’s χ2 (1) P OR

Child Gender 1.3 .59 5.1 .02* 3.8

Child Ethnicity .88 .88 1.0 .31 2.4

Child Primary Language -.90 .79 1.3 .26 .41

Age at time of assessment (months) -.06 .04 2.4 .12 .94

Household type (single-parent vs. two-parent) -.01 1.3 .00 .99 .99

Household income -1.2 .69 3.3 .07 .29

Frequency of literacy activities 1.2 .38 10.2 < .01** 3.3

*p<.05. **p<.01. Note. 10 children are siblings. Analyses were conducted excluding these children and 

results did not differ.

Model χ2 = 19.52, p = .01

Pseudo R2 = .25

n = 117