FINAL	EVALUATION	REPORT	

Cincinnati Museum Center

January	2014	
                                         

                       

           

FINAL EVALUATION REPORT 

Cincinnati Museum Center 

January 2014 

Acknowledgements 

This report was developed under grant #10SIHOH001

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. Upon request, this

material will be made available in alternative formats for people with disabilities.
 

 

 

 

Social Innovation Fund Year One Review 

Cincinnati Museum Center 

Early Childhood Science Inquiry Training for Educators 

(ECSITE) 

1. Program Theory and Objectives 

Cincinnati Museum Center (CMC) is home to the Cincinnati Museum Center’s Duke Energy 

Children’s Museum which has been serving the educational needs of young children, their caregivers 

and early childhood education providers for over 10 years. As part of their commitment to addressing 

educational needs in their community, CMC has joined local efforts to improve young children’s school 

readiness and eventually meet goals for kindergarten assessments (85% of children meet or exceed 

cut-off scores for readiness by 2020: United Way 2010). Currently, roughly 40% of children in the 

region served by the ECSITE program (Cincinnati, OH, Newport and Covington, KY) have school 

readiness assessment scores below these benchmarks (Successby6 2010, United Way 2010).1 

Consistent with national trends (Wat 2008), children from lower-income families are unprepared at 

higher rates. The ECSITE program was designed with the intent to reach those children most at risk for 

falling short of readiness standards through the articulation of evidence-based practices for teachers, 

parents, their children, and the coordination of CMC and community resources with Head Start early 

childhood education classrooms. 

To accomplish these aims, ECSITE relies on an approach often described as “sciencing” (e.g. 

Harlan & Rivkin, 2000; Neuman, 1972, 1992), which encourages the integration of science experiences 

across other curriculum domains through an enriched environment, planned lessons, and incidental 

conversations with children. This strategy has demonstrated long-term, beneficial impact on children’s 

school readiness skills (Conezio & French 2002; Esach & Fried 2005; Greenfield, Jirout, Dominquez, 

Greenberg, et al. 2009). The potential of science inquiry approaches for accelerating preschool 

development and school readiness skills, particularly for children from economically disadvantaged 

households who attend Head Start programs, is also indicated by research (Peterson & French, 2008; 

French 2004). 

Sciencing strategies rely foremost on teachers trained in this approach. Because preschool 

teachers may at first feel intimidated or overwhelmed by the idea of integrating science inquiry across 

classroom activities (PNC Banks, Grow Up Great with Science Teacher Survey 2009; Wat 2010), 

ECSITE works to actively educate, encourage, and support participating teachers. The centerpiece of 

the program therefore, is a year-long professional development program. This consists of an 

introductory seminar on science inquiry instructional practices followed by a series of workshops. At 

these workshops, teachers apply what they’ve learned and demonstrate science inquiry lessons to their 

peers and museum instructors who offer supportive feedback. These workshops include a Science PotLuck in which teachers meet to exchange lesson plans and provide feedback. The workshops and 

1 The cut-off scores set by the community for school readiness include a result of 14 or greater on the Ohio state mandated 

Kindergarten Readiness Assessment – Literacy, and in Kentucky, a normal curve equivalent score of 50 or more on the 

Developmental Indicators for the Assessment of Learning 3 (DIAL 3). 

1 | P a g e 
 

 

 

 

 

 

 

 

 

 

trainings are supplemented with materials and other resources supplied by CMC including field trip 

opportunities and on-site visits from a mobile “Program-On-Wheels.” Because classroom mentorship 

has been linked to more favorable child developmental outcomes (Epstein, 1993), teachers who 

successfully complete a year of training, are asked to continue as mentors in the program. These 

“teacher trainers” work together with CMC instructors/facilitators on-site, in the classrooms, to support 

new teachers trained by ECSITE, ensuring the continuity and sustainability of the program. 

Because parent engagement is also crucial to classroom success and increased school 

readiness (e.g. Henderson & Mapp, 2002; Foster, Lambert, Abbott-Shim, McCarty, & Franze, 2005; 

Reynolds, Mavrogenes, Berzruczko & Hagemann, 1996), the ECSITE program extends CMC 

resources to children’s families through open field-trip opportunities including Family Science Night and 

the Learning Through Play Conference. The Learning Through Play Conference (LTP) is unique in the 

region, bringing together families and teachers in workshops, interactive play opportunities and 

lectures, designed to enrich understanding of preschool development and school readiness. In addition, 

CMC provides ECSITE participating classrooms with take-home family activity packets which 

encourage parent-child interaction and discussion around science-inquiry based projects. 

To strengthen the impact of preschool teacher professional development and parent 

engagement efforts on children’s preschool development and school readiness, the ECSITE project 

provides tools, supports, and classroom supplies that sustain school readiness objectives. In their 

classrooms, and in their take-home materials, children access a variety of exploration tools. A range of 

science manipulatives including droppers, tongs, and graduated cylinders not only encourage fine 

motor skills, but invite children to test, measure, and investigate concepts of quantity, volume and 

cause and effect. Live animal habitats are also provided to encourage curiosity and observation skills. 

Children may encounter these materials throughout the classroom, but particularly at math and science 

centers improved through the ECSITE program. Presentations offered through Program-On-Wheels 

align with Ohio and Kentucky curriculum standards while the also emphasize science inquiry lesson 

objectives. 

The relationship between ECSITE program components and their anticipated outcomes is 

summarized below in their program’s logic model (Figure 1). By consistently meeting program 

implementation goals, the ECSITE program ultimately expects to: a.) improve teacher confidence and 

competency in science education and the classroom integration of science inquiry strategies, b.) 

increase levels of parent participation and family engagement in science and math based activities, c.) 

strengthen each child’s pre-academic skills as indicated both by measures of preschool development 

and finally, d.) improve scores on kindergarten assessments of school readiness. 

2. Summary of Program Evaluation Methodology 

Evaluation of the ECSITE program focuses on assessing both program implementation and its 

preliminary outcomes. Program activities and outputs outlined by the logic model describe the 

program’s intended implementation. In academic years 2011-12 and 2012-13, the ECSITE program 

was led by CMC staff and is currently offered to preschool teachers, families and their children enrolled 

at Head Start classrooms with the Hamilton County Community Action Agency (CAA) and the Northern 

2 | P a g e 
3 | P a g e 

Figure 1: ECSITE program logic model 
 

 

 

Kentucky Community Action Commission (NKCAC). This includes a total of 41 classrooms across two 

of CAA’s locations (19 in academic year 2011-12 and 34 classrooms in academic year 2012-13) as well 

as a total of 20 classrooms at NKCAC. To obtain the necessary materials and training to deploy 

science inquiry strategies across the curriculum, the program requires that teachers participate in a 

minimum of 20 hours of professional development (PD) and receive 22 hours of classroom mentoring 

either from CMC facilitators or teacher trainers who have already completed ECSITE training. Hours of 

participation in PD are monitored by CMC facilitators. In addition, each teacher was asked to complete 

a survey at the beginning and end of their year-long PD participation to monitor changes in their level of 

comfort with science instruction and the integration of science inquiry strategies in their classrooms. 

The survey also gauges the frequency with which teachers access CMC and other community 

resources as a part of science inquiry lessons. CMC, in partnership with the Public Library of Cincinnati 

and Hamilton County, Barnes and Noble, and PNC bank, provide classroom supplies and family takehome packets to participating teachers over the course of their PD. These materials are deployed for a 

minimum of 8 ECSITE designed science inquiry based lessons. In addition, teachers help plan and 

participate in Family Science Nights and field trips and many attend the LTP conference. The utilization 

of these community and CMC resources, participation and planning of ECSITE events is tracked by 

CMC facilitators. 

Families participate in take-home activities, receive free admission to CMC for family science 

events, and receive free vouchers to attend the LTP conference. Levels of family participation in 

ECSITE events will be monitored by CMC facilitators who will track receipts of vouchers and free 

exhibit passes. Families may also participate in an interest survey at ECSITE events to offer feedback 

on take-home activities and materials. 

Near the beginning and end of each academic year, children attending CAA classrooms in Ohio 

are assessed to monitor changes in preschool development using the Bracken Basic Concepts Scale 

(BBCS) (Pearson, 2007). This testing occurs in October and March of each year to allow insight into 

changes in preschool development shaped by the ECSITE program. Children are administered a total 

of 8 subtests. This includes the first five subtests which make up the school readiness composite (SRC) 

and assesses concept awareness related to colors, letters, numbers and counting, sizes and 

comparisons, and shapes. Additional subtests relevant to science and math knowledge and awareness 

are also being administered. These are the subtests for texture and material (subtest 8), quantity 

(subtest 9), and time and sequence (subtest 10). The Bracken has well documented validity and 

reliability with high rates of intercorrelation with other measures of preschool development and school 

readiness, including the Kindergarten Readiness Assessment – Literacy (KRA-L). The KRA-L is the 

Ohio mandated assessment of school readiness that is administered to kindergarten students in 

October each year. 

Children enrolled in NKCAC classrooms as assessed annually using the Developmental 

Indicators for the Assessment of Learning 3 (DIAL 3). This assessment evaluates developmental 

abilities across five domains including motor, language, concepts, self-help, and emotional skills. 

Although they are not directly comparable, the domains of knowledge evaluated with this tool, 

especially receptive concepts and language, overlap with those assessed by the Bracken. These areas 

include color and shape identification, counting, meaningful counting, concepts, and problem solving. 

Preschool results of the DIAL3 are frequently compared with results of the same test administered as a 

4 | P a g e 
 

school readiness assessment in many school districts in Northern Kentucky. A re-test using the DIAL 3 

has been implemented in kindergarten classrooms in lieu of a state mandated school readiness 

assessment. 

Where permitted by signed parental consent and appropriate data sharing agreements between 

school districts, children who participated in ECSITE classrooms were tracked into kindergarten and 

their preschool assessments were matched to the results of their school readiness assessment (in 

Ohio, the KRA-L, and in Kentucky, the DIAL 3). 

The evaluation will employ both within and between group analyses to answer questions of 

program impact. There are two cohorts for each of the groups described in Table 1, corresponding to 

two separate years of SIF funding for the ECSITE program. Performance on preschool assessments 

will be compared within groups, between these cohorts to examine improvements in scores as a 

function of teacher’s increased experience in the program. In addition, ECSITE participants’ 

performance on assessments of school readiness will be compared with a similar group of at-risk 

children without program exposure, both with and without preschool experience. Descriptive details are 

offered for groups of parents and teachers, although currently, there is no access to a comparison 

group of non-ECSITE teachers and parents and so these cannot be incorporated into the evaluation 

plan. 

Table 1: Groups Identified for Program Evaluation 

Group 

Name Cohort Composition and Size 

Cohort 1 (academic 

year 2011-12): 

CAA (Ohio) teachers trained through the ECSITE program: 

34 new teachers 

Group 1a Cohort 2 (academic 

year 2012-13): 

CAA (Ohio) teachers trained through the ECSITE program: 

32 new teachers and 

8 teachers as trainers with previous program experience. 

Cohort 1 (academic 

year 2011-12): 

NKCAC (Kentucky) teachers trained through the ECSITE program: 

6 new teachers 

Group 1b Cohort 2 (academic 

year 2012-13): 

NKCAC (Kentucky) teachers trained through the ECSITE program: 

6 new teachers and 

6 teachers as trainers with previous program experience. 

Group 2 

Cohort 1 (academic 

year 2011-12): Parents and families of children enrolled in ECSITE classrooms in 

both Ohio and Kentucky 

Total cohort sizes unknown. Cohort 2 (academic 

year 2012-13): 

Cohort 1 (academic 

year 2011-12): 

388 Children assessed in ECSITE participating CAA classrooms 

(Ohio) – Data source is the SuccessBy6® Database 

Group 3a Cohort 2 (academic 

year 2012-13): 

576 Children assessed in ECSITE participating CAA classrooms 

(Ohio) – Data source is the SuccessBy6® Database 

Two-Year 

Participants 

91 Children assessed in ECSITE participating CAA classrooms 

(Ohio) – Data source is the SuccessBy6® Database 

Group 3b Cohort 1 (academic 

year 2011-12): 

Children assessed in ECSITE participating NKCAC classrooms 

(Kentucky) – Data source is the Northern Kentucky Early Childhood 

5 | P a g e 
 

Cohort 2 (academic 

year 2012-13): 

Database. 

Information pending approval of data sharing agreement. Signed 

Two consent forms for 174 children in cohort 1. -Year 

Participants 

Group 4a 

Cohort 1 (academic 

year 2011-12): 

122 Children graduated from ECSITE participating CAA classrooms 

(Ohio) who are enrolled in kindergarten and received a school 

readiness assessment (KRA-L) in academic year 2012-13. - Data 

Source is the Cincinnati Public Schools Database. 

Cohort 2 (academic 

year 2012-13): 

Children graduated from ECSITE participating CAA classrooms 

(Ohio) who are enrolled in kindergarten and received a school 

readiness assessment (KRA-L) in academic year 2013-14: - Data 

pending, KRA-L not yet available. - Data Source is the Cincinnati 

Public Schools Database. 

Group 4b 

Cohort 1 (academic 

year 2011-12): 

Children graduated from ECSITE participating NKCAC classrooms 

(Kentucky) who are enrolled in kindergarten and received a school 

readiness assessment (DIAL 3) in academic years 2012-13 and 

2013-14: - Information pending approval of data sharing agreement 

– Data source is the Northern Kentucky Early Childhood Database. 

Cohort 2 (academic 

year 2012-13): 

Cohort 1 (academic 

year 2011-12): 

840 ECSITE eligible children from non-participating Head Start 

classrooms in Ohio who are enrolled in kindergarten and received a 

school readiness assessment (KRA-L) in academic year 2012-13. ­

Data Source is the SWELL Regional Survey. Group 5a 

Cohort 2 (academic 

year 2012-13): 

ECSITE eligible children from non-participating Head Start 

classrooms in Ohio who are enrolled in kindergarten and received a 

school readiness assessment (KRA-L) in academic year 2013-14: ­

Data pending, KRA-L not yet available. - Data Source is the SWELL 

Regional Survey. 

Cohort 1 (academic 

year 2011-12): 

ECSITE eligible children from non-participating Head Start 

classrooms in Kentucky who are enrolled in kindergarten and 

received a school readiness assessment (DIAL 3) in academic years 

2012-13 and 2013-14: Information pending approval of data sharing 

agreement. – Data source is the Northern Kentucky Early Childhood 

Database. 

Group 5b Cohort 2 (academic 

year 2012-13): 

Group 6a 

Cohort 1 (academic 

year 2011-12): 

462 ECSITE eligible children with no documented preschool (Ohio) 

who are enrolled in kindergarten and received a school readiness 

assessment (KRA-L) in academic year 2012-13. - Data Source is the 

SWELL Regional Survey. 

Cohort 2 (academic 

year 2012-13): 

ECSITE eligible children with no documented preschool (Ohio) who 

are enrolled in kindergarten and received a school readiness 

assessment in academic year 2013-14: - Data pending, KRA-L not 

yet available. - Data Source is the SWELL Regional Survey. 

Group 6b 

Cohort 1 (academic 

year 2011-12): 

ECSITE eligible with no documented preschool (Kentucky) who are 

enrolled in kindergarten and received a school readiness 

assessment (DIAL 3) in academic years 2012-13 and 2013-14: 

Information pending approval of data sharing agreement. – Data 

source is the Northern Kentucky Early Childhood Database. 

Cohort 2 (academic 

year 2012-13): 

6 | P a g e 
 

 

 

 

The 2011-12 academic year was CAA’s inaugural year in the ECSITE program. Eight of the 32 

teachers trained in the program this first year became teacher trainers in the 2012-13 academic year. 

NKCAC had participated in the ECSITE program since its inception in academic year 2009-2010. In 

academic year 2011-12, NKCAC had 6 new teachers enrolled in ECSITE professional development 

and 6 teacher trainers. In academic year 2012-13, NKCAC expanded their teacher trainers to 8. This 

means that across both academic years a total of 72 new preschool teachers received ECSITE 

professional development. 

Our understanding of Group 2 (Parents and families of children enrolled in ECSITE classrooms) 

is evolving as we receive more information about their needs and interests through their voluntary 

participation in ECSITE events and interest surveys. A review of event receipts and a recent 

participation survey distributed at Family Science Day suggested a rate of parent participation of 

approximately 15% (n = 87) in academic year 2011-12. While this group, in theory, includes all of the 

families with children enrolled in ECSITE classrooms, in practice, this group is represented by those 

parents who either attended or provided responses to surveys solicited at ECSITE events. To preserve 

their anonymity on these surveys, the demographic composition and background experiences for this 

group have not been shared with the evaluators. 

Data agreements with ECSITE classrooms at NKCAC are still pending at the time of this report. 

While NKCAC has now provided consent to the evaluator to access information for children with signed 

consent, the Northern Kentucky Early Childhood Database (NKECD) at Northern Kentucky University is 

still processing the data request. As a result, no demographic information or assessment data is 

available for children in Kentucky ECSITE classrooms at this time. 

In addition, only limited demographic data is currently available for children currently enrolled in 

ECSITE classrooms. CAA is currently reviewing requests for additional data that can provide additional 

context for children’s developmental assessment scores (Table 2). 

Table 2: Available Demographics for Children Enrolled at CAA 

Group 

Group 3a 

Cohort 

1 (2011-12) 

2 (2012-13) 

Two-Year 

Participants 

No Gender 

Data 

2.8% 

(11) 

2.4% 

(14) 

0.0% 

(0) 

Females 

50.0% 

(194) 

46.7% 

(269) 

44.0% 

(40) 

Males 

47.2% 

(183) 

50.9% 

(293) 

48.4% 

(44) 

Total 

Number 

Assessed 

388 

576 

91 

More extensive demographic information is available for graduates of ECSITE classrooms who 

have been matched with their KRA-L score and for preliminary comparison groups 5a and 6a extracted 

from blinded data available from the Cincinnati Public Schools database and SWELL regional survey 

database (Tables 3a – 3c). 

7 | P a g e 
 

–

Table 3a: Demographics for CAA ECSITE Graduates and Comparison Groups: Gender 

Group 

Group 4a 

Group 5a 

Group 6a 

Cohort 

1 (2011-12) 

1 (2011-12) 

1 (2011-12) 

No Gender 

Data 

2.4% 

(3) 

1.5% 

(13) 

0.9% 

(4) 

Females 

52.5% 

(64) 

48.3% 

(406) 

51.1% 

(236) 

Males 

45.1% 

(55) 

50.1% 

(421) 

48.1% 

(222) 

Total 

Number 

Assessed 

122 

840 

462 

Table 3b: Demographics for CAA ECSITE Graduates and Comparison Groups: Race 

Group 

Group 

4a 

Group 

5a 

Group 

6a 

Cohort 

1 

1 

1 

No Data 

0.8% 

(1) 

9.0% 

(76) 

8.7% 

(40) 

African 

Am. 

86.1% 

(105) 

35.4% 

(297) 

17.5% 

(81) 

Asian 

0.0% 

(0) 

0.2% 

(2) 

0.2% 

(1) 

Hispanic/ 

Latino 

7.4% 

(9) 

6.2% 

(52) 

6.7% 

(31) 

Multi/ 

Other 

2.5% 

(3) 

4.5% 

(42) 

6.5% 

(30) 

Caucas. 

3.3% 

(4) 

44.2% 

(371) 

60.4% 

(279) 

Total 

122 

840 

462 

Table 3c: Demographics for CAA ECSITE Graduates and Comparison Groups: Language 

Group 

Group 4a 

Group 5a 

Group 6a 

Cohort 

1 (2011-12) 

1 (2011-12) 

1 (2011-12) 

No Data 

77.0% 

(94) 

2.3% 

(20) 

1.5% 

(7) 

English 

19.7% 

(24) 

92.5% 

(777) 

95.9% 

(443) 

Spanish 

2.5% 

(3) 

22.1% 

(27) 

2.6% 

(12) 

Other 

0.8% 

(1) 

13.1% 

(16) 

0.0% 

(0) 

Total 

122 

840 

462 

3. Evaluation of Program Implementation 

A summary of program implementation questions, their place within the logic model, and their 

findings are summarized in the chart below (Table 4). In most cases, implementation goals for high 

levels of teacher, parent and child participation, easy availability of materials and resources, and 

consistent replication in the integration of science inquiry approaches across multiple classroom 

domains were met. 

In years one and two, all ECSITE teachers received an additional hour of training beyond the 20 

hours originally planned for the program. In addition 66 classrooms received materials and furnishing 

through ECSITE, 3 classrooms more than originally anticipated. A high rate of parent participation in 

Family Science Night is noted in museum attendance records. Although only a small percentage of 

families voluntarily submitted a survey, a large majority (92%) indicated spending time with their 

children on take-home science projects, and most (85%) felt that it was either “very easy” or “somewhat 

easy” to engage in science conversations at home. All ECSITE teachers reported increased exploration 

of science centers within their classrooms by children. 

8 | P a g e 
 

ECSITE looks forward in the year ahead to increasing involvement by teachers in the Learning 

Through Play conference and finding new strategies for understanding the needs and interests of 

families with children enrolled in ECSITE classrooms. 

Table 4: Summary of Program Implementation Questions and Findings 

1.) To what extent are teachers participating in program activities? 

Logic Model OP1: 

Teachers participate in professional development, mentoring and implement/use tools 

provided. 

Logic model Component Findings 

a.) Teachers participate in 20 hours of 

professional development. 

CAA and NKCAC teachers each participated in a 

total of 21 hours of professional development at 

Cincinnati Museum Center (seven 3-hour 

workshops). 

b.) Teachers receive 22 hours of mentoring in 

science inquiry techniques. 

Both CAA and NKCAC teachers received at least 22 

hours of mentoring from one of our three mentors. 

c.) Teachers receive classroom inquiry 

materials/furnishings for 63 rooms total (43 at 

CAA 17 in year 1 and 28 in year 2 and 20 at 

NKCAC). 

A total of 66 rooms received classroom inquiry 

materials/furnishings (CAA=46; 32 year one; 14 

year two; NKCAC=20) 

d.) Teachers implement workshop science tools 

for a minimum of 8 ECSITE lessons. 

Teachers implemented considerably more than 

eight science lessons. Of the 22 CAA classrooms 

that returned their post-surveys this year, 21 

classrooms presented AT LEAST one lesson from 

each of the 8 topics. 

e.) Teachers participate in planning Family 

Science Day and Science Night. 

Each CAA classroom at the two primary centers 

(excluding six remote home visitation classrooms) 

provided an activity at the Family Science Days. 

CAA Teacher Trainers collaborated on the takehome activity provided to families at the Science 

Night at Museum Center. 

f.) Teachers participate in LTP conference. 

Up to 26 teachers used a total of 95 available slots 

for professional development sessions at the 

Learning Through Play conference. 

2.) To what extent are families participating in program activities? 

Logic Model OP2: 

Parents participate in interactive and family -based learning 

a.) Parents receive free museum admission 

and attend family science night and other 

activities. 

A total of 225 adults and 450 children participated in 

the Family Science Night at Museum Center, which 

provided free admission to Duke Energy Children’s 

Museum and the Museum of Natural History & 

Science. 

b.) Parents participate in Take-home activities. 

100% of families attending Family Science Night 

received a dinosaur-related take-home activity. 

92% of the 87 families returning our parent survey 

reported spending time with their children on at least 

one take-home activity. 

c.) Parents participate in science related family 

conversations. 

Of the 87 families who returned the parent survey, 

85% noted that it was “very easy” or “somewhat 

9 | P a g e 
 

-

 

 

 

 

 

easy” to participate in science conversations with 

their children. 

d.) Parents receive free vouchers and 

participate in LTP conference. 

50 vouchers were provided to families (good for up 

to five family members/session). 

3.) To what extent are children participating in program activities? 

Logic Model OP3: 

Children participate in science and math inquiry based activities 

a.) Children participate in classroom-based 

enrichment activities. 

100% of classrooms reported an increase in 

children’s exploration of science content and 

participation in science enrichment activities. 

b.) Children participate in science museum 

interactions. 

450 children visited the Museum of Natural History 

& Science during Family Night, and all classrooms 

were offered Field Trips to visit Duke Energy 

Children’s Museum and the Museum of Natural 

History & Science. 

c.) Children participate in Family Science Day 

and Family Science Night. 

Science activities were offered by all classrooms at 

the Family Science Day, and 450 children attended 

Family Science Night at Cincinnati Museum Center. 

4. Evaluation of Program Outcomes 

A summary of program outcome questions, their place within the logic model, and their findings 

are summarized in the chart below (Table 5). In many cases, outcome goals for increased rates of 

teacher comfort and competency in the preparation of science and math lessons, high rates of parent 

involvement and participation in transition services, appropriate developmental progress for children, 

and enrollment in quality preschool were met or exceeded. 

Of particular note, a large majority of parents participating in the survey (88%) reported that they 

believe science is either “very important” or somewhat important for their children’s school readiness. In 

addition, there were significant increases in teacher’s math and science confidence from the beginning 

to the end of each academic year. Detailed results for the confidence survey are listed in Table 6. 

Assessments of preschool development for Group 3a indicated that many more than half of 

ECSITE participating children with both pre- and post- tests improved their performance on the school 

readiness composite across the school year. The largest increases in scores were observed for those 

children who were enrolled in ECSITE classrooms for two years. The Bracken assessment was 

administered to children according to planned project and evaluation methodology. The total number of 

children with both pre- and pots- tests is summarized in Table 7. A summary of their mean scaled 

scores for the School Readiness Composite and three additional subtests (Texture and Material, 

Quantity and Time and Sequence) is presented in Table 8. Statistical comparison of Bracken 

performance for Group 3a cohorts is described in Table 9. 

Table 5: Summary of Program Outcome Questions and Findings 

1.) Do preschool teachers report increased comfort and competency related to early 

science learning? 

Logic Model OT1, Teacher: 

Competence in science inquiry and knowledge in children 

Logic model Component Findings 

10 | P a g e 
 

Teachers report increased comfort and 

competency in knowledge related to early 

science learning. 

Increase in positive responses on teacher surveys 

from the beginning of training to the end of training. 

An increase in comfort and competency in both 

teachers in their first AND second years in the 

program was reported. For teachers in their first 

year of the program, pre-surveys indicated that 70% 

of them were “confident” or “very confident” in 

teaching early science learning. That number 

increased to 95% feeling “confident” or “very 

confident” on the post-survey (only one respondent 

listed him/herself as “a little unsure”). 

For teachers in their second year in the program, 

pre-surveys indicated that 84% of them were 

“confident” or “very confident” in teaching early 

science learning. That number increased to 94% 

feeling “confident” or “very confident” on the postsurvey (only one respondent listed him/herself as “a 

little unsure). 

2.) Do parents engage in age-appropriate science related activities with their children? 

Logic Model OT1, Family: 

Competence in science inquiry and knowledge in children 

a.) Parents report being comfortable engaging 

in science conversations and activities with 

their child 

Parents report greater comfort on survey. 85% of 

parents reported that it was “very easy” or 

“somewhat easy” to have conversations with their 

children about science. 

b.) Parents report being supportive of their 

child's development of science inquiry and 

school readiness 

Parents report increased awareness and support on 

survey. 88% of parents report that they believe that 

science is “very important” or “somewhat important” 

for their children’s kindergarten readiness. 

3.) Does increased preschool teacher competency in early childhood science inquiry and 

preschooler’s increased contact with science and math curriculum improve these 

children’s knowledge of math and science concepts on normed preschool 

assessments? 

Logic Model OT1, Child: 

Competence in science inquiry and knowledge in children 

a.) Increase on the Bracken School Readiness 

Composite and other subtests for Hamilton 

County ECSITE participants. 

In academic year 2011-12, 69.3% of children in 

Group 3a, cohort 1, maintained or surpassed their 

pre-test scaled score on the Bracken school 

readiness composite. 

In academic year 2012-13, 75.4% of children in 

Group 3a, cohort 2, maintained or surpassed their 

pre-test scaled score on the Bracken school 

11 | P a g e 
 

 

 

 

 

readiness composite. 

70.3% of two year participants in ECSITE, Group 

3a, two years, saw positive gains in scaled scores 

across post-tests in both academic years. 

b.) Increase on the DIAL-3 for Northern 

Kentucky ECSITE participants Awaiting approval of data sharing agreement. 

4.) Is science inquiry fully integrated in preschool curricula and are children engaged in 

learning? 

Logic Model OT2: 

Science inquiry is fully integrated in the preschool curricula and children are engaged in 

learning 

a.) Most teachers receive ECERS science 

scores of 5 or higher for their classroom 

environment. 

There was no site visit this past year from 

independent evaluator, Open Minds. 

b.) Parents/caregivers report spending time on 

conversations and activities with child. 

92% (N=87) of adults report participating in 

conversations with their children regarding science 

topics. 

c.) Increased interest in science/math 

classroom areas. 

100% of classrooms reported an increase in 

children’s exploration of science content and 

participation in science enrichment activities. 

Table 6: Pre/Post-Survey Data Comparison – Teacher Confidence Survey 

Cohort 

Cohort 

1 Pre 

Cohort 

1 Pre 

Cohort 

1 Post 

Cohort 

1 Post 

Cohort 

Cohort 

2 Pre 

Cohort 

2 Pre 

Cohort 

2 Post 

Cohort 

2 Post 

Rating and 

Percent 

Confident 

for Subject 

Average 

Rating 

% Fairly or 

Confident 

Average 

Rating 

% Fairly or 

Confident 

Rating and 

Percent 

Confident 

for Subject 

Average 

Rating 

% Fairly or 

Confident 

Average 

Rating 

% Fairly or 

Confident 

Language 

3.3 

95% 

3.6 

100% 

Language 

3.4 

96% 

3.5 

100% 

Arts 

3.5 

95% 

3.6 

100% 

Arts 

3.5 

96% 

3.5 

100% 

Science 

3.1 

84% 

3.7 

94% 

Science 

2.9 

70% 

3.5 

95% 

Motor 

3.5 

95% 

3.7 

94% 

Motor 

3.4 

96% 

3.5 

100% 

Math 

3.3 

89% 

3.6 

94% 

Math 

3.1 

87% 

3.4 

100% 

Soc/Emo 

3.3 

95% 

3.7 

100% 

Soc/Emo 

3.3 

96% 

3.4 

100% 

Literacy 

3.6 

95% 

3.6 

88% 

Literacy 

3.5 

91% 

3.5 

100% 

Ratings: 1-Not at all, 2-A little, 3-Fairly, 4-Very 

12 | P a g e 
 

- -

-

-

 

-

- - -

Table 7: Assessment Data Available for Individual Sub-Groups. 

Group 

Group 3a 

Cohort 

1 (2011-12) 

2 (2012-13) 

Two-Year 

Participants 

Total Number 

Pre Tests 

310 

390 

77* 

Total Number 

Post Tests 

303 

483 

91* 

Both Pre and 

Post Tests 

225 (58.0%) 

297 (51.6%) 

55* (60.4%) 

Total Number 

Assessed 

388 

576 

91** 

*Two year participants with both pre-tests, post-tests and all pre- and post- assessment data in both
 

2011-12 and 2012-13 respectively.
 

**Two year participants with both post-tests.
 

Table 8: CAA Performance on the Bracken 2011-12 and 2012-13 

CAA 2 Year 

Performance on 

Bracken 

2011 12 2012 2013 Two Year Participation 

N=225 N=297 N=91 

PreTest PostTest Difference PreTest PostTest Difference First 

PostTest 

2011-

12 

Second 

PostTest 

2012-

13 

Difference 

School 

Readiness 

Composite 

Avg Raw 

Score 

32.3 

(18.2) 

43.3 

(19.2) 

10.99*** 31.3 

(18.9) 

45.9 

(18.4) 

14.6*** 31.1 

(17.7) 

51.7 

(17.0) 

20.6*** 

Avg 

Percentile 

29.4 

(28.6) 

33.7 

(28.3) 

4.32*** 28.7 

(27.4) 

37.5 

(27.7) 

8.8*** 33.1 

(30.7) 

37.7 

(29.2) 

4.6* 

Avg 

Scaled 

Score 

7.7 

(3.2) 

8.1 

(3.2) 

.47*** 7.7 

(3.0) 

8.6 

(3.0) 

.9*** 8.1 

(3.3) 

8.5 

(3.3) 

.4 

Texture 

and 

Material 

Avg Raw 

Score 

7.5 

(5.6) 

10.5 

(6.1) 

2.9*** 7.7 

(5.2) 

10.9 

(5.8) 

3.1*** 7.2 

(5.0) 

13.4 

(6.1) 

6.2*** 

Avg 

Percentile 

27.5 

(24.9) 

30.3 

(24.4) 

2.8* 29.8 

(24.5) 

31.3 

(23.4) 

1.5 29.4 

(24.9) 

35.5 

(27.2) 

6.1* 

Avg 

Scaled 

Score 

7.6 

(2.7) 

8.0 

(2.6) 

.37* 7.9 

(2.6) 

8.2 

(2.3) 

.24 7.9 

(2.6) 

8.5 

(2.7) 

.62* 

Quantity Avg Raw 

Score 

8.7 

(6.6) 

12.3 

(7.5) 

3.6*** 8.3 

(6.3) 

12.3 

(6.9) 

4.0*** 7.7 

(6.2) 

14.0 

(7.0) 

6.3*** 

Avg 

Percentile 

22.7 

(21.1) 

26.6 

(23.6) 

3.9** 22.9 

(20.3) 

26.0 

(20.8) 

3.1** 22.4 

(21.6) 

25.3 

(22.1) 

2.9 

Avg 

Scaled 

Score 

7.2 

(2.4) 

7.5 

(2.6) 

.34* 7.3 

(2.2) 

7.6 

(2.3) 

.31* 7.2 

(2.4) 

7.5 

(2.4) 

.31 

Time and 

Sequence 

Avg Raw 

Score 

5.8 

(4.6) 

8.1 

(5.1) 

2.3*** 5.8 

(4.6) 

8.2 

(4.7) 

2.4*** 5.0 

(4.5) 

9.5 

(5.1) 

4.5*** 

Avg 

Percentile 

30.5 

(24.9) 

32.0 

(24.0) 

1.4 31.1 

(24.8) 

32.3 

(23.3) 

1.3 29.1 

(26.2) 

21.0 

(23.6) 

1.9 

Avg 

Scaled 

Score 

7.9 

(2.7) 

8.2 

(2.5) 

.30 8.0 

(2.8) 

8.3 

(2.3) 

.3* 7.6 

(3.0) 

8.2 

(2.2) 

.59 

Note: Values in table are means with standard deviations in parentheses. 

*p < .05, **p < .01, ***p < .001 

Post hoc analyses of paired t-tests indicating improvement in mean scaled scores frequently 

revealed low power and small effect sizes. However, contrasts in SRC for Group 3a, cohort 2, yielded 

statistically significant results exceeding 99% power and an effect size greater than .4. It is tempting to 

13 | P a g e 
 

 

-

 

infer that this improvement in SRC scores for Group 3a, cohort 2, may be due to longer teacher 

participation in ECSITE trainings, although additional analyses will be required (Table 9). 

Table 9: Significance of Change in Mean Subtest Scaled Scores by Evaluation Sub-Group 

Sub Groups 3a SRC Texture and 

Material 

Quantity Time and 

Sequence 

Cohort 1 

(2011-12) 

N=225 

Mean diff. .47 .37 .34 2.7 

s.m.e. .12 .15 .14 .18 

t 3.8 2.5 2.5 1.7 

p <.001 .01 .02 .10 

Power 70.2% 49.9% 48.7% 49.7% 

Effect size .26 .17 .16 .11 

Cohort 2 

(2012-13) 

N=297 

Mean diff. .92 .24 .31 .30 

s.m.e .12 .15 .13 .15 

t 7.72 1.60 2.4 2.0 

p <.001 .11 .02 .05 

Power 99.9% 48.9% 22.3% 51.7% 

Effect size .45 .09 .09 .12 

Cohort: Two 

Year 

Participants 

(2011-2013) 

N=91 

Mean diff. .37 .62 .31 .39 

s.m.e .21 .27 .25 .31 

t 1.8 2.3 1.2 1.9 

p .07 .03 .22 .06 

Power 99.9% 51.1% 51.2% 49.9% 

Effect size .45 .24 .13 .20 

s.m.e. – Standard mean error
 

Cells highlighted in gray illustrate statistically significant increase in mean scaled score below p=.05.
 

Cells highlighted in yellow illustrate post Hoc power analyses greater than 80% and moderate to large 

effect sizes.
 

Table 10 indicates the percentage of children with both pre- and post- test who improved their 

percentile rank and who improved by at least one performance category. The Bracken Assessment 

designates 5 performance categories as follows: “Very Advanced” (scaled scores 16-19), “Advanced” 

(scaled scores 13-15), “Average” (8-12), “Delayed” (scaled scores 5-7), and “Very Delayed” (scaled 

scores 1-4). 

A graphic representation of change in School Readiness Composite performance category for 

Group 3a, cohort1 is provided in Figure 2, and for Group 3a, cohort 2, in Figure 3. Change in 

performance category is significantly correlated for both academic years [Group 3a, cohort 1, rs (225) = 

.778, p<.001; Group 3a, cohort 2, rs (297) = .709, p<.001]. Change in performance category for twoyear program participants is also significantly correlated [Group 3a, Two Years, rs (91) = .781, p<.001]. 

Greater improvement in SRC scores was achieved in the second year of the ECSITE program at CAA. 

14 | P a g e 
 

-

- -

 

 

 

 

Table 10: Percent of Children with Improved Percentile Rank and Performance Category 

Sub 

Test 

SRC 

8 

9 

10 

Percent of Children with Pre and Post tests 

Who Maintained or Improved their 

Percentile Rank 

Cohort 1 

(2011-12) 

N=225 

Cohort 2 

(2012-13) 

N=297 

Cohort 3 

(Two-Year) 

N = 91 

44.4% 54.9% 70.3% 

46.7% 43.1% 58.2% 

43.1% 48.1% 59.3% 

49.3% 43.1% 62.6% 

Percent of Children who Improved by an 

Entire Performance Category 

Cohort 1 

(2011-12) 

N=225 

Cohort 2 

(2012-13) 

N=297 

Cohort 3 

(Two-Year) 

N = 91 

24.9% 29.6% 22.0% 

25.8% 23.9% 31.9% 

28.8% 27.6% 23.1% 

29.1% 22.9% 29.7% 

Cells highlighted in yellow indicate highest percent increase. 

Figure 2: Change in SRC Performance Category 

Group 3a, Cohort 1 (2011-12) 

Number of chldren with pre- and 

post- test SRC Bracken 

120 

100 

80 

60 

40 

20 

0 

Very Delayed Delayed Average Advanced Very Advanced 

Pre 2011 Post 2012 

39 33 

80 65 

86 

105 

19 20 1 2 

Figure 3: Change in SRC Performance Category 

Group 3a, Cohort 2 (2012-13) 

15 | P a g e 

Number of chldren with pre- and 

post- test SRC Bracken 

200 

150 

100 

50 

0 

41 30 

114 

79 

121 

159 

19 29 2 0 

Very Delayed Delayed Average Advanced Very Advanced 

Pre 2012 Post 2013 
 

5.) Preliminary and Baseline Analysis of Impacts 

A summary of program impact questions, their place within the logic model, and their findings 

are summarized in the chart below (Table 11). A total of 122 children with some record of attendance at 

a CAA center were successfully tracked into kindergarten. For ECSITE participants with Bracken 

assessment data, there were 18 children form Group 3a, cohort 1, who could be successfully matched 

to KRA-L scores in 2012-13 (Table 12). The lower than expected rates of matching appear to be due to 

the following three factors: 1.) High rates of residential and school mobility which prevented assessment 

of all children enrolled throughout the academic year at CAA; 2.) Enrollment in school districts other 

than Cincinnati Public Schools; 3.) Low rates of signed releases from CAA families providing 

permission to link assessment data. Additional efforts to secure signed parent permissions should help 

to increase the number of matches in future years of evaluation. 

Table 11: Summary of Program Impact Questions and Findings 

1.) Do children achieve school readiness and developmental success in kindergarten? 

Logic Model Impact: 

Children achieve developmental success and school readiness in kindergarten 

a.) Hamilton County children who participated 

in ECSITE are better prepared in math/science 

than children who did not participate in science 

enrichment , and demonstrate increased 

kindergarten readiness because of this 

preparation as indicated by KRA-L scores. 

An estimated 75 children in academic year 

2012-13 and 115 children in academic year 

2013-14 will be matched to school readiness 

scores. They will be more likely to achieve 

scores above community benchmark standards 

indicating school readiness. 

122 Children previously enrolled in CAA ECSITE 

classrooms were linked to KRA-L scores. Their 

mean was 17.43 (s.d. 7.26) 

18 Children previously enrolled in ECSITE were 

matched to KRA-L scores. Their mean raw score 

was 16.22 (s.d. 7.26) 

b.) Northern Kentucky children who participated 

in ECSITE are better prepared in math/science 

than children who did not participate in science 

enrichment , and demonstrate increased 

kindergarten readiness because of this 

preparation as indicated by DIAL-3 scores. 

An estimated 75 children in academic year 

2012-13 and 115 children in academic year 

2013-14 will be matched to school readiness 

scores. They will be more likely to achieve 

scores above community benchmark standards 

indicating school readiness. 

Awaiting approval of data sharing agreement 

16 | P a g e 
 

- -

Mean KRA-L Score 

25 

20 

15 

10 

5 

0 

17.4 16.2 

19.6 

15.8 

Group 4a - Graduates Group 4a.1 - Group 5a - Non- Group 6a - Eligible 

of ECSITE Graduates of ECSITE ECSITE Head Start Participant with No 

(N=122 - s.d.7.16) with Bracken Scores (N=840 - s.d.6.74) Documented 

(N=18 - s.d.7.26) Preschool 

(N=462 - s.d.6.56) 

- -

-

-

c.) Children will be ready for kindergarten in 

Hamilton County. 35.3% of ECSITE participant with KRA-L scores 

received a score of 19 or above. > 60% of ECSITE participants score a 19 or 

above on the KRA-L. 

Table 12: Children Enrolled in ECSITE and Tracked into Kindergarten 

Total Number of 

Children Tracked 

from ECSITE into 

Kindergarten 

Total with 

Bracken 

Assessment Data 

Total Number 

Pre Tests* 

Total Number 

Post Tests* 

Both Pre and 

Post Tests* 

122 18 17 16 15 

*Bracken assessments for the previous academic year (2011-12) matched to KRA-L scores. 

In Ohio, the Kindergarten Readiness Assessment-Literacy (KRA-L) scores provide insight into 

ECSITE participants’ success in kindergarten (Figure 4). The differences between these groups is 

significant [ F (3, 1438) = 32.82, p<.001]. 

For ECSITE graduates with Bracken scores, 2011-12 Bracken SRC scaled post-test scaled 

scores were significantly correlated with KRA_L scores [ rp(15) = .519, p = .04]. Of the 18 children with 

Bracken data who were tracked into kindergarten, 16 had post-test scores. Their KRA-L mean is slightly 

higher than the total KRA-L mean for this sub-group. A comparison of their Bracken sub-test results and 

mean KRA-L score is provided in Table 13. 

Figure 4: KRA-L Scores for Children Tracked Into Kindergarten 

Table 13: Comparison of Bracken Post-Test Scaled Score Means and Mean KRA-L 

(Sub ) 

Test 

N = 16 

SRC 

7.19 

(3.21) 

8 

8.44 

(2.50) 

9 

7.56 

(2.87) 

10 

7.50 

(1.54) 

KRA L 

16.7 

(6.68) 

17 | P a g e 
 

 

 

6.) Summary and Next Steps 

ECSITE has met or exceeded many of the goals they have set for implementation and 

outcomes for their program. ECSITE teachers are receiving quality professional development and 

classroom materials which contribute to increased levels of teacher comfort and competency in science 

and math lessons, increased engagement of children in the classroom with science and math activities, 

and positive trends in parent support and participation. 

While data is still forthcoming for children enrolled at NKCAC in Northern Kentucky, data 

provided for children enrolled in ECSITE classrooms at CAA (Ohio) indicate substantial pre- to posttest improvement in measures of preschool development. The largest increases in Bracken scores can 

be seen for children in the second year of the ECSITE program and for children with two years of 

experience. This may be related to wider integration of ECSITE materials and teaching practices after 

the inaugural year of the program at CAA. 

Because of the tight correlation in Bracken and KRA-L scores demonstrated for Group 3a, 

cohort 1, the substantial improvements in Bracken results for children in the second year of ECITE at 

CAA suggest a positive trend in KRA-L scores will emerge over time. Although the KRA-L scores for 

cohort 1 were lower than those for children from other Head Start agencies who did not receive the 

benefits of the ECSITE program (Group 5a), difficulties in tracking children into kindergarten may have 

unrealistically slanted these results. Provided there are improvements in collection of parent 

permissions and in tracking children into kindergarten, future trends in mean KRA-L scores for 

subsequent cohorts of CAA graduates may be more indicative of program outcomes. 

Given the initial successes of the program in years one and two, the fidelity of program 

implementation, and positive trends in children’s developmental outcomes, we anticipate highly positive 

outcomes in the data for year 3 as second year and two-year participants in ECSITE are promoted into 

kindergarten. 

18 | P a g e 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

References 

Conezio, K. & French, L. (2002). Science in the preschool classroom: Capitalizing on children’s 

fascination with the everyday world to foster language and literacy development. Young 

Children. 12-18. 

Epstein, A.S. (1993). Training for quality: Improving early childhood programs through systematic 

inservice training (Monographs of the High/Scope Educational Foundation, No. 9). Ypsilanti, 

MI: High/Scope Press. 

Eshach, H. &Fried, M. (2005). Should science be taught in early childhood? Journal of Science 

Education and Technology. 14(3). 315-335. 

Foster, M.A., Lambert, R., Abbott-Shim, M., McCarty, F., Franze, S. (2005). A model of home 

learning environment and social risk factors in relation to children’s emergent literacy and 

social outcomes. Early Childhood Research Quarterly, 20, 13-36.\ 

French, L. (2004). Science as the center of a coherent, integrated early childhood curriculum. Early 

Childhood Research Quarterly, 19. 138-149. 

Greenfield, D.B., Jirout, J. ,Dominguez, X., Greenberg, A., et al.(2009). Science in the preschool 

classroom: A programmatic research agenda to improve science readiness. Early Education 

and Development. 20(2). 238-264. 

Harlan, J.D. & Rivkin, M.S. (2000). Science Experiences for the early childhood years: An integrated 

approach. Upper Saddle River, NJ: Prentice Hall. 

Henderson, A.T. & Mapp, K.L. (2002). A new wave of evidence: The impact of school, family, and 

community connections on student achievement. Austin, TX: Southwest Educational 

Development Laboratory. 

Kellogg Foundation. (2004).W.K. Kellogg Foundation logic model guide. Retrieved from 

http://www.wkkf.org/knowledge-center/resources/2010/Logic-Model-DevelopmentGuide.aspx. 

Neuman, D. (1972). Sciencing for young children. In K.R. Baker (Ed.), Ideas that work with young 

children. Washington, D.C.: National Association for the Education of Young Children. 

Neuman, D. (1992). Experiences in science for young children. Prospect Heights, IL: Waveland 

Press. 

Ohio Department of Education. (n.d.). Academic Content Standards. Retrieved from 

http://ims.ode.state.oh.us/ODE/IMS/ACS/Grades_ContentAreas/Default.asp. 

Peterson, S.M. & French, L. (2008). Supporting young children’s explanations through inquiry 

science in preschool. Early Childhood Research Quarterly, 23. 395-408. 

19 | P a g e 
 

 

Reynolds, A.J., Mavrogenes, N.A., Bezruczko, N., Hagemeann, M. (1996). Cognitive and familysupport mediators of preschool effectiveness: A confirmatory analysis. Child Development, 

67, 1119-1140. 

SuccessBy6, United Way of Greater Cincinnati. (2012). KRA-L annual report for academic year 2011-

12. 

United Way of Greater Cincinnati. (2010). The state of the community: A report on the socio-economic 

health of the greater Cincinnati region. 4th Edition. 

Wat, A. (2008). The Pre-K Pinch: Early education and the middle class . Pew Charitable Trusts, State 

and Consumer Initiatives. Downloaded from: http://www.pewstates.org/projects/pre-k-now328067 

Wat, A. (2010). The Case for Pre-K in Education Reform: A summary of program evaluation findings. 

Pew Center on the States. Downloaded from http://www.pewstates.org 

20 | P a g e