Inkster Family Literacy Movement 

Year Three Evaluation Report 

October 26, 2015 

Prepared by 

Public Sector Consultants Inc. 

Lansing, MI 

Prepared for 

Starfish Family Services 

Inkster, MI 
1 







Evaluation Overview

INTRODUCTION 

The Starfish Family Services Inkster Family Literacy Movement (IFLM) is a United Way of Southeastern 

Michigan (UWSEM) Social Innovation Fund (SIF) project. The primary focus of the IFLM is to positively 

affect the language development, cognitive development, and communication skills of young children in 

Inkster through an innovative mix of community engagement, community campaigns, and a coordinated 

continuum of family literacy programs, services, and events. Starfish Family Services aims to decrease the 

percentage of children exhibiting vulnerabilities in these two areas, which aligns well with (1) UWSEM’s

overarching SIF project goal of having 80 percent of Inkster children begin kindergarten ready to succeed 

by 2018, and (2) Starfish’s broader vision that “all Inkster children start school ready to succeed.” 

The IFLM is unique in that it uses a number of methods to reach both individual residents and the broader 

community. For example, it runs a public awareness campaign to educate all Inkster residents about the 

importance of early literacy; it works with a coalition of community partners to promote literacy-enhancing 

practices and programs; it hosts family-friendly events throughout the community to demonstrate early 

literacy activities that children and families can use at home; and it provides more intensive programming 

for many parents and young children to further encourage early literacy. 

Through its many efforts, the IFLM strives to 







improve coordination among community agencies focusing on early literacy activities, 

improve community residents’ knowledge of and attitudes and beliefs about the importance of early 

literacy and how to promote it among children, and 

increase the proportion of Inkster children who enter school ready to learn. 

While the overall SIF project is undergoing evaluation at the national level, the national evaluating 

organization also requires that each individual SIF project funded by the UWSEM be evaluated at the local 

level. To that end, the IFLM contracted first with a team from Wayne State University (WSU) to evaluate 

years one and two of the project, and then contracted with Public Sector Consultants (PSC) to evaluate 

years three and four. This report, prepared by PSC, focuses specifically on year three activities, data 

collection, and results. 

YEAR THREE EVALUATION ACTIVITIES 

Completing the IFLM year three evaluation was challenging, particularly due to severe time constraints 

caused by delays in approval of the Subgrantee Evaluation Plan (SEP) and the Institutional Review Board 

(IRB) application. Following is a description of key year three activities and the impact they had on the 

overall evaluation project. 

In October 2014 (the end of the second program year), the IFLM hired PSC to replace the outgoing 

evaluation team from WSU. As soon as it was on board, PSC began working diligently with the IFLM, 

the UWSEM, and community partners to review, update, and enhance the project’s existing evaluation 

plan. 

In December 2014, WSU delivered its Year Two Evaluation Report. PSC immediately reviewed the 

report and worked quickly to finalize its own recommendations for improving and completing the 

evaluation in years three and four. 

In February 2015, PSC, UWSEM staff, and IFLM staff learned that the SEP needed to be updated and 

re-submitted to the SIF evaluation team. At the heart of this request was the need to provide moderate 
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

2

evidence in addition to the preliminary evidence that was presented in previous years. To that end, PSC 

worked with the UWSEM and the IFLM to further revise the evaluation approach and submitted an 

updated SEP in March 2015. It was approved in August 2015. 









While the SEP was being reviewed and approved by the SIF evaluation team, PSC submitted an IRB 

application to the Michigan Public Health Institute (as required by the SIF). IRB approval was received 

in September 2015. 

In September 2015, PSC began collecting preliminary evidence through (1) the Knowledge, Attitudes, 

Beliefs, and Behaviors (KABB) survey, which was administered to IFLM participants; (2) focus groups 

with IFLM families; (3) individual interviews with community members; and (4) post-event surveys 

carried out by IFLM staff at the conclusion of each community event. However, since PSC could not 

begin collecting data until approvals of the SEP and IRB were received, the data collection timeframe 

was severely shortened. This, in turn, limited the amount of data that PSC could gather, analyze, and 

use for evaluation purposes. 

The UWSEM technical staff was responsible for providing moderate evidence related to the impact of 

the IFLM project on kindergarten readiness. It used a quasi-experimental design of the Children 

Observation Record (COR) Advantage assessment (a birth-to-kindergarten assessment that assists 

teachers in supporting children at every developmental level, including those who are English language 

learners and/or have special needs), which is administered to all children enrolled in Head Start through 

Starfish Family Services. Using propensity score matching (a three-group, paired comparison), the 

UWSEM technical staff compared the COR Advantage scores of IFLM children with those of Inkster 

and non-Inkster children to determine whether IFLM activities and marketing efforts improved 

kindergarten readiness in the community. This analysis was completed in October 2015. 

In October 2015, PSC received notification that funding would not be available for year four of the 

IFLM project, making the year three report the last one produced for purposes of evaluating the IFLM. 

The year three report includes only year three project results from the internal evaluation (program 

metrics and post-event surveys) conducted by IFLM staff and the external evaluation (KABB surveys, 

focus groups, interviews, and COR Advantage data) conducted by PSC, with assistance from UWSEM.

The SEP called for two full years of data collection, but only a one-month data collection window was 

possible due to delays in IRB approval. As a result, the data and analysis provided in this report are limited 

in the following ways: 







The KABB survey was not administered to community partners and the wider Inkster community as 

planned. The data set, therefore, is limited to only those IFLM participants who were surveyed 

electronically and on site at IFLM events in September 2015. 

PSC’s analysis of KABB survey data is based on 41 responses, which is not nearly as robust a sample 

as planned. Because of the low number of responses, the survey data should be treated as anecdotal. 

Because different data collection methods were used, comparisons cannot be made between the KABB 

surveys conducted by PSC and those completed in years one and two. 
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

3

Internal Evaluation Results 

PROGRAM METRICS 

The following metrics and targets were established by IFLM program staff specifically for year three: 

















Establish three Book Nooks by the end of the program year 

Distribute 6,000 books to Inkster children and families (500 each month) by the end of the program 

year 

Distribute 50 percent of the books to Inkster children aged 0–5 by the end of the program year 

Have 500 parents/caregivers participate in IFLM events by end of the program year (42 per month) 

Have 600 Inkster children participate in IFLM events by the end of the program year (50 per month) 

Have 30 percent of IFLM participants attend three or more program events or activities 

Maintain 13 formalized agreements with community partners to support IFLM events and activities 

Fill 100 volunteer slots at IFLM events and activities 

As shown in Exhibit 1, the program met seven of the eight year three targets. A total of 299 

parents/caregivers (unduplicated count) participated in IFLM activities, along with 555 Inkster children 

(unduplicated count). 

EXHIBIT 1. Performance on Program Metrics, Year Three 

Metric

Monthly

Avg. TOTAL

Target

Met/Unmet

Three Book Nooks established by the end of the year n/a 1 UNMET

6,000 books distributed to Inkster children and families (500 each month) 538 6,456 MET

50 percent of book distribution goes to Inkster children aged 0–5 n/a 59% MET

500 parents/caregivers participate in Inkster Family Literacy Movement events by 

end of the program year (42 per month) *

104 1,246 MET

600 Inkster children participate in Inkster Family Literacy Movement events by the 

end of the program year (50 per month) *

72 863 MET

30 percent of IFLM participants attend three or more program events or activities n/a 30% MET

Maintain 13 formalized agreements with community partners to support IFLM events 

and activities

n/a 16 MET

Fill 100 volunteer slots at IFLM events and activities 17 186 MET

* Participation is calculated each month. Thus participants who attend in a prior month were counted as 

new participant for the subsequent month. 

POST-EVENT SURVEYS 

After IFLM events and activities, IFLM staff distributed a post-event survey to assess overall satisfaction 

with the event. A total of 291 surveys were returned by participants in year three. Ninety-eight percent 

(98%) of respondents reported they were either very satisfied (95 percent) or somewhat satisfied (3 percent). 

At the same time, nearly everyone said they would definitely (89 percent) or probably (10 percent) 

recommend an IFLM event to a friend or family member. 
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

4

When asked to rate certain aspects of each event, 98 percent of participants either strongly agreed (78 

percent) or agreed (20 percent) that the event(s) they attended (1) increased their child’s/children’s 

awareness of the importance of reading, (2) increased their own awareness of the importance of reading, 

and (3) gave them ideas about how they could promote reading with their own child/children. 

In addition, the surveys asked participants to indicate how they planned to use what they learned in the 

weeks following the event. As shown in Exhibit 2, a majority of respondents planned to engage in all of the 

actions tested, with “Read to my child” being the most prevalent response at 91 percent. 

EXHIBIT 2. Planned Actions among IFLM Participants, Year Three 

Action %

Read to my child 91

Have my child read to me 82

Talk to my child about his/her day 81

Play a reading or word game with my child 73

Talk to a friend/family about the importance of reading activities 64

External Evaluation Results 

PRELIMINARY EVIDENCE 

In the original evaluation design, PSC was to provide preliminary evidence by collecting data through 

multiple KABB surveys, focus groups, and interviews. As noted earlier, data collection was limited to a 1-

month collection period (September 2015). Following is a description of each of the methods PSC used, 

the number of times and the populations with which those methods were used, and an analysis of the results. 

KABB Survey 

The KABB survey was designed to (1) assess the knowledge, beliefs, and feelings of respondents about 

literacy; (2) document literacy-promoting activities that occur in households with children, and (3) obtain 

a broader understanding of how respondents are involved in the Inkster community. The proposed 

evaluation design included using the KABB survey instrument with a representative sample of Inkster 

residents and with attendees at community coalition partner events in order to compare the results with 

those of surveyed IFLM participants. However, because of delayed approvals of the SEP and the IRB 

application, surveys were only conducted with IFLM participants. 

A total of 41 surveys were collected in September 2015. Survey respondents were primarily African 

American (65 percent) and female (97 percent); a large majority had either completed some college (32 

percent) or had a college degree (53 percent); and about half (52 percent) had an annual household income 

of less than $25,000. A frequency report of survey responses is included as an appendix to this report. 

Because of the low number of responses, PSC recommends treating the KABB survey data as anecdotal. 

Three sections of the KABB survey assess the knowledge, beliefs, and feelings of respondents by asking 

them to indicate the extent to which they agree or disagree with statements related to early childhood 

literacy. Each question was answered on a 5-point Likert scale, ranging from strongly agree to strongly 

disagree. For some of the questions, a positive response (agree or strongly agree) indicated a correct 

understanding of the best way to promote early childhood literacy. For others, a negative response (disagree 
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

5

or strongly disagree) indicated a correct understanding of the best way to promote literacy in young 

children. An overall score was calculated for each section of the survey, as well as for the three sections 

combined. If the respondent answered strongly agree or agree when a positive response was called for or 

strongly disagree or disagree when a negative response was appropriate, their response was coded as a one, 

otherwise it was a zero. Using this approach, each of the three sections of the survey had a total of six 

possible points (one for each question), for a total possible KABB score of 18. 

Among survey respondents, there was a mean KABB survey score of 14 (out of a possible 18). As shown 

in Exhibit 3, mean scores were highest on the “knowledge” section (5.9), indicating that, on average, 

respondents are quite knowledgeable about specific activities that support literacy in young children. On 

average, respondents appropriately agreed or disagreed with “belief” statements 4.5 times out of 6, and with 

“feeling” statements only 3.7 times out of 6. These scores indicate there is still more work to be done related 

to shaping adults’ understanding of how best to encourage early childhood literacy. 

EXHIBIT 3. KABB Scores among IFLM Participants, Year Three 

Section

Mean Score

(6 points

possible)

Knowledge 5.9

Beliefs 4.5

Feelings 3.7

The fourth section of the KABB survey was designed to identify the extent to which respondents (1) were 

familiar with and/or had participated in any IFLM events, and (2) engaged in other early literacy activities 

with their children. 









Participation: The majority of respondents (64 percent) were either very familiar (32 percent) or 

somewhat familiar (32 percent) with the programs and activities provided by the IFLM. The majority 

of respondents attended a Family Literacy Night (71 percent), received a book from the IFLM (57 

percent), or went to a Reading Rally (55 percent). On average, respondents reported participating in 3.3 

of the eight IFLM activities listed in the survey, indicating that while they attended some events, they 

did not attend the majority of them. 

Time spent reading: Over half of respondents (58 percent) reported reading just about every day with 

their child (or children) in the past week, with 25 percent reading three or four times with them, 11 

percent reading once or twice, and 6 percent not reading at all during that timeframe. About two-thirds 

of respondents (64 percent) reported that their child enjoys being read to for 10–30 minutes in one 

sitting and one-fifth (22 percent) said their child enjoys being read to for only a few minutes at a time. 

No respondents reported that their child enjoys being read to for more than an hour, and just three 

percent said their child does not enjoy being read to at all. 

Early literacy activities at home: When asked which activities they engaged in with their child at 

home over the past week, respondents most frequently said they told their child a story (79 percent); 

involved him/her in such household chores as cooking, cleaning, setting the table, or caring for pets (76 

percent); sang him/her songs or listened to music (74 percent); and played with toys or games indoors 

(74 percent). 

Early literacy activities in the community: Respondents were also asked what types of activities they 

engaged in with their children in the community during the past month. The most common responses 
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

6

included visiting a playground, visiting a park, or going on a picnic (76 percent); going to a mall or 

store, like Target (71 percent); attending a church activity or church school (64 percent); and attending 

an event sponsored by a school, community, ethnic, or religious group (62 percent). 

PSC also conducted analyses (called Pearson Correlations1) to see if there were statistically significant 

correlations between the scores of individual respondents on the KABB survey and four variables: (1) 

familiarity with the IFLM, (2) time spent reading with children, (3) engaging in early literacy activities at 

home, and (4) engaging in early literacy activities in the community. PSC found that there was no 

statistically significant correlation between KABB survey scores and any of the tested variables. However, 

there was 







a statistically significant correlation (alpha=0.05) between the number of IFLM activities participated 

in and the amount of time spent reading with children (r=.355), 

a statistically significant correlation (alpha=0.05) between the number of early literacy activities 

conducted at home in the past week and the amount of time spent reading with children (r=.416), and 

a statistically significant correlation (alpha=0.01) between the number of early literacy activities 

conducted at home in the past week and the number of activities attended in the community in the past 

month (r=.769). 

Focus groups 

PSC conducted three focus groups in September 2015 with twenty-five people from Inkster who had 

participated in IFLM activities over the past year. Many participants had attended several IFLM activities. 

The activities mentioned most often included Reading Rallies, Family Literacy Nights at McDonalds, free 

book distributions, and book clubs for both children and adults. Most focus group participants said they had 

heard about these events when participating in other Starfish programs, from Starfish staff, and from flyers 

available at Starfish and other community locations, such as libraries. Many participants said they sought 

out the events as social opportunities for themselves and their children, and stated a desire to identify new 

ways to positively engage their children that did not include staring at a television, computer, or handheld 

screen. 

At all of the IFLM activities, families were shown literacy and math games they could replicate at home. 

Participants reported that some of the IFLM activities, such as FLOW (Family Literacy Opportunity 

Workshop), taught them strategies to help their children learn how to read or how to be stronger readers. 

Since participating in the IFLM activities, participants reported that they now have much larger libraries at 

home, and that they and their children are reading more often. Participants also said they are (1) talking 

about reading and books more often with other family members; (2) encouraging their children to read more 

often; and (3) reading more often with their children, including with their older children who already know 

how to read. Some parents, including those in the book club, indicated that they are reading more themselves 

at home, which, in turn, encourages other family members to read. 

Participants reported that they loved the variety of activities offered at IFLM events. For example, in 

addition to engaging in games and activities, participants reported seeing visiting authors, storytellers, a 

poetry café, an opera singer, and a living museum with African American historical figures played by local 

community members. This variety, they said, keeps the events interesting and exposes children to new 

things they may want to learn more about in the future. Participants also said they like the large number 

 

1

 The Pearson Correlation statistic (r) is a measure of the strength and direction of a relationship between two 

variables. Statistically significant correlations at either 95 percent (alpha=0.05) or 99 percent (alpha=0.01) 

confidence levels are presented as a number between -1 and +1. Stronger correlations are closer to -1 and +1, while 

weaker correlations are closer to zero. 
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

7

and variety of books provided through the program. Again, having numerous options to choose from allows 

everyone to find a book they like, and enables parents to find books that their children may not have chosen, 

but that they really enjoy. 

Some participants identified challenges with the IFLM activities, and others offered recommendations for 

improving events in the future. For example, some members said that the Reading Rallies and Family 

Literacy Night events were too crowded. As a result of space limitations, children were often herded from 

one activity to the next instead of being allowed to fully engage in the offerings, and some children could 

not participate in certain activities at all. Participants also reported that some parents in the community are 

unable to read themselves and, thus, may be too embarrassed to come to a literacy event or activity. With 

regard to improvements, some participants suggested that there should be more activities for older children 

rather than focusing predominantly on young children, and others recommended providing transportation 

assistance to enable more people to engage in the different IFLM activities. 

Everyone stated that they would recommend IFLM activities to others, and many said they had already 

done so or had previously taken family members with them to events. One person recommended that

funders should attend the activities and events to see how successful they are and how much families need 

them. 

Individual Interviews with IFLM Coalition Members 

PSC attempted to interview representatives of all seven coalition partner organizations: the Inkster Public 

Library, the Inkster Task Force, the Josie Odum Morris Literacy Project, Sisters Who Are Powerful 

(SWAP), Starfish Family Services, the Taylor Reading Corps, and Westwood Community Schools. 

However, PSC was only able to reach four coalition members for interviews: one from Westwood 

Community Schools, one from the Inkster Public Library, and two from the Josie Odum Morris Literacy 

Project. While all of the organizations have been involved in the IFLM Coalition since its inception, two of 

the representatives interviewed were new, having been involved with the coalition for less than a year. 

Interviewees were asked questions about the coalition’s mission, its successes, concerns about the coalition 

and the challenges it has faced, partners who should be invited to join the coalition, and the coalition’s 

future. 







Mission: Those interviewed had varying interpretations of the coalition’s mission. While some 

respondents explained its goal as improving literacy in Inkster, others said it was simply to promote 

literacy and increase access to available literacy materials. One person reported that its mission was to 

ensure that each of the partner organizations’ did not hold events at the same time to avoid competition 

for attendees. 

Success: Coalition members viewed the IFLM events they participated in as successful. They noted 

that event attendees had a good time and, thus, were more inclined to attend future events. They also 

said that participants all received literacy materials. 

Concerns: Interviewees reported a couple of concerns about the coalition and its work. 





Two coalition members reported that few people in the community know which organizations 

comprise the IFLM. According to one interviewee, many people in the community think the IFLM 

is the early childhood literacy division of Starfish Family Services as opposed to a broader coalition 

of community organizations. It was suggested that using all the coalition partners’ logos could serve 

as an educational opportunity and increase commitment to participating in and assisting with IFLM 

events among partners. 

Interviewees reported wanting to do more to increase community awareness about the importance 

of literacy and improve literacy rates in Inkster. Additionally, coalition members reported that 

although many books and other literacy materials were distributed to families, not enough was done 
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

8

to teach people, especially adults, how to read, or to connect them with organizations/resources that 

could teach them. 











Challenges: Coalition members reported that the lack of continuity of coalition membership and the 

dissolution of Inkster Public Schools (IPS) were the greatest challenges facing the IFLM. 

The Leanna Hicks Public Library, one of the coalition partners in Inkster, has had two different 

directors in the last two years and is currently without a director. The IFLM itself has had three 

different program managers. 

IPS was a member of the coalition until the district was dissolved in 2013. Inkster students now 

attend Westwood Community Schools, Wayne-Westland Public Schools, Romulus Public Schools, 

Taylor Public Schools, or an area charter or private school. Only Westwood Community Schools 

is an IFLM Coalition member. In addition to being disruptive, the dissolution of IPS makes it 

difficult to measure the impact the IFLM may be having on student literacy because (1) the IFLM 

has to reach out to every district to obtain data, and (2) it is challenging for schools to break out 

data only for those students who live in Inkster. 

Partners: Each of the four interviewees made a different recommendation regarding who should join 

the coalition. One person said the library should be more involved; one person said the surrounding 

school districts should be more active; one person recommended that Western Wayne Health Center 

join the coalition; and another person advocated for the involvement of parents and older children. The 

latter individual argued that having parents and youth on the coalition could generate more ideas for 

literacy activities for older children. 

Continuation: Interviewees expect the coalition to continue working to improve literacy in Inkster, but 

some believe not all of the current members will remain involved. One person noted that without 

funding, coalition-planned events would change dramatically. 

MODERATE EVIDENCE 

As required by the SEP, moderate evidence of impact on kindergarten readiness is provided through the 

quasi-experimental design of the COR Advantage assessment, which is given to all children enrolled in 

Head Start through Starfish Family Services. Using propensity score matching (a three-group, paired 

comparison), COR Advantage scores were compared between IFLM children, Inkster children, and nonInkster children to help determine if IFLM activities and marketing efforts improved kindergarten readiness 

in the community. 

COR Advantage 

As a preliminary step in the analysis we screened all of the COR advantage items to check that the scores 

were normally distributed and met all the assumptions required for subsequent analyses. The data were 

found to meet all the distributional and statistical requirements for the analyses to be conducted and no 

transformations or adjustments were needed. 

We then computed propensity scores in order to create matched controls for the individuals in the 

intervention group. The use of propensity scoring methods has been expanding in program evaluation as it 

provides an alternative to randomized control trials which are often, untenable to implement (Guo & Fraser, 

2013). Propensity scores are conditional probability estimates of receiving treatment, given a set of 

measured covariates. This can be expressed as a binary logistic regression of the form, P(Ti|Xi=x), where 

P is the probability of person i receiving treatment T, given that person i has a specific set of values x on 

the vector of conditioning or covariate variables X. These covariate variables X are potential confounds 
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

9

related to self-selecting into treatment, which might be an alternative explanation for mean differences in 

the outcome for those receiving treatment and those in a control group. 

Using this approach, an individual propensity score can be calculated in the form of a logit, or log odds, 

such that loge(P/1-P) yields a log odds (or logit estimate) of the likelihood of receiving treatment—given 

the values of the covariates for that person. Once these logits are calculated, individuals in the treatment 

group and the control group are matched based on their propensity scores, or their individual logit values. 

Since these values are usually not exactly the same, a method is needed to determine how close a control 

propensity score needs to be to a treatment propensity score in order to be considered a match. 

We used a nearest neighbor-matching procedure which involves calculating the smallest absolute difference 

between any two pairs of individuals’ propensity scores among all the treatment and control group pairs. 

Any difference that falls within the range of that minimum distance is considered a match. 

Propensity scores were calculated based on the following covariates: family income, age of parents, age of 

participating child, parent education, primary language used in the home, and ethnicity. Propensity scoring 

and matching procedures were computed using the statistical software package R. 

Once the propensity scores were established and matching had been completed baseline scores for all 34 

domains of the COR Advantage were compared between the intervention group and the matched controls 

using an independent sample t-test. These analyses indicated that there were no significant differences 

between the control group and the intervention group on any of the COR Advantage domains at baseline. 

To assess the primary evaluation question regarding the improvement in COR Advantage scores for IFLM 

relative to controls we conducted separate mixed methods ANOVAs for each of the 34 COR Advantage 

domains. In these ANOVAs intervention status was specified as the between subjects factor and the 3 

repeated measurements (baseline, post, and follow-up) were specified as the within subjects factor. 

Specifically, in these ANOVA models, we tested to see if there was a significant group (intervention v. 

control) by time (baseline, post-test, follow-up) interaction. Because of the disparity in sample size between 

the intervention and control groups (see exhibit 4), unequal within group variances were assumed. These 

analyses indicated that for 8 of the 34 COR Advantage domains the IFLM participants showed statistically 

greater gains than the propensity score matched controls. These domains were: Reflection – F(2,259)=2.58, 

p=.07, η2= .02; emotional expression and regulation F(2,259)=3.08, p=.04, η2= .024; positive relationships 

with peers F(2,259)=2.61, p=.07, η2= .02; gross motor skills F(2,259)=4.18, p=.01, η2= .03; use of personal 

care and healthy behaviors F(2,259)=2.64, p=.07, η2= .02; pre-reading and print awareness skills 

F(2,259)=3.28, p=.03, η2= .03, use of predictive reasoning and drawing conclusions F(2,259)=3.89, p=.02, 

η

2= .03, and self-concept / theory of mind skills F(2,259)=4.34, p=.01, η2= .04. In 4 additional domains 

IFML participant showed marginally significant gains in comparison to propensity score matched controls. 

These domains included: attachment/positive adult relationships, auditory verbal comprehension, 

movement and creative expression, and understanding concepts of past, present, and future. These 

marginally significant findings are important given that the intervention sample size was relatively small. 

EXHIBIT 4. Sample sizes for treatment v. control groups across three waves of COR 

assessment 

COR Assessment Wave

Treatment 

Group n

Control 

Group n

Total Sample 

N
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

10

Wave 1 (baseline) 17 374 392

Wave 2 (Post Intervention) 18 356 374

Wave 3 (Follow-up) 17 341 358

Conclusion 









The IFLM met its targets for programming. With the exception of establishing Book Nooks, the 

IFLM met all of its targets for book distribution, events, and community member participation in 

Inkster. 

IFLM activities and events get high marks from participants. Post-event surveys show that IFLM 

activity participants were overwhelmingly satisfied (98 percent) with their experience. These high 

marks were echoed by focus group participants who raved about the quality and variety of activities 

provided by IFLM staff. Focus group participants credited IFLM staff, events, and activities with giving 

them the skills, ideas, and strategies they need to encourage literacy at home. 

IFLM participants are highly engaged with literacy. According to post-event surveys, the majority 

of respondents planned to do all of the literacy activities tested, with the vast majority of participants 

indicating that they plan to read to their children (91 percent). At the same time, over half (58 percent) 

of IFLM KABB survey respondents said they already read to their child/children just about every day. 

Finally, there is a positive correlation with participation in IFLM activities and the amount of time spent 

reading with children. 

Coordination with partners could be improved. Because no KABB survey data were collected from 

participants in events independently hosted by coalition partners, no conclusions can be made about 

their familiarity or engagement with the IFLM. Individual interviewees indicate some concerns with 

the coordination of efforts among coalition members, ranging from logo use on printed materials to 

worries about the broader reach and impact of activities on participant knowledge and literacy. At the 

same time, changes in IFLM staff and volatility in partner organizations (for example, the Inkster Public 

School District was dissolved during the project) also made coordination difficult. 

As stated above, the data collected and presented in this year three report are only a fraction of what was 

proposed and approved in the SEP. The small sample of IFLM KABB surveys collected demonstrate some 

positive correlations between IFLM participation, the amount of time spent reading with children, and 

broader engagement with other learning activities and events in the community, so it appears that the IFLM 

could be having a positive impact on literacy-promoting behaviors. However, in the absence of KABB 

survey data from community members and coalition partner participants, it is not possible to make 

comparisons to these other groups. As a result, the evaluation is lacking critical context for triangulating 

data, which is makes it difficult to draw conclusions about the preliminary evidence provided through 
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

11

surveys, focus groups, and interviews. In other words, while PSC is able to document and verify that the 

IFLM met nearly all of its program metrics, it cannot say with confidence what impact these efforts had on 

the community. 

Finally, data from year two indicated that IFLM participants who also participated in Starfish Head Start 

scored higher on kindergarten readiness assessments, as did Inkster children, when compared to non-Inkster 

Head Start children. Mixed methods ANOVAs were conducted with propensity score matched intervention 

v. control group designation as the between group factor and data collection wave (pre, post, follow-up) as 

the within group factor on each of the 34 domains of the COR Advantage data in year three. These results 

indicated that on 8 of the 34 COR Advantage domains the IFLM participants showed statistically greater 

gains than the propensity score matched controls. These domains were: Reflection, emotional expression 

and regulation, positive relationships with peers, gross motor skills, use of personal care and healthy 

behaviors, pre-reading and print awareness skills, use of predictive reasoning and drawing conclusions, and 

self-concept / theory of mind skills. In 4 additional domains IFML participant showed marginally 

significant gains in comparison to propensity score matched controls. These domains included: 

attachment/positive adult relationships, auditory verbal comprehension, movement and creative expression, 

and understanding concepts of past, present, and future. These marginally significant findings are important 

given that the intervention sample size was relatively small. 
12

Appendix 

IFLM KABB Survey 

[N=41] 

AWARENESS OF IFLM 

1. Overall, how familiar are you with the programs and activities provided by the Inkster Family Literacy Movement 

(IFLM)? 

2. Which of these IFLM programs have you and/or your children participated in during this past year? [CHECK 

ALL THAT APPLY] 

a) Reading Rally 55%

b) Family Literacy Night at McDonalds 71

c) Literacy Chat 7

d) Book Nook 5

e) Paws Event 7

f) Received a book from the IFLM 57

g) Book Club 41

h) IFLM Community Impact Awards 14

i) Josie Odum Morris Literacy Program (JOMLP) Literacy Extravaganza 21

j) Play Group 31

k) Poetry Café 14

l) Other 11

a) Very unfamiliar 24%

b) Somewhat unfamiliar 10

c) Neither familiar or unfamiliar 2

d) Somewhat familiar 32

e) Very familiar 32
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

13

3. How have you heard about IFLM? [CHECK ALL THAT APPLY] 

a) Information delivered to your home (flyers, brochures, etc.) 21%

b) Information you got at a community event (flyers, brochures, etc.) 69

c) Facebook (saw information or announcements from IFLM) 29

d) Twitter (saw tweets from IFLM) 0

e) Website (visited the IFLM website) 19

f) Email message (got an email about IFLM) 52

g) IFLM Literacy Vine newsletter 10

h) None of these (never heard of IFLM prior to taking this survey) 7

4. Have you ever seen any pictures of the character “Jay”?

a) Yes 75%

b) No 25

5. Have you ever seen an IFLM billboard like the one pictured here around Inkster? 

a) Yes 60%

b) No 40
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

14

KNOWLEDGE, BELIEFS, AND FEELINGS 

6. Tell us how much you agree or disagree with each of the following statements: 

Statement

Strongly 

Agree Agree Neither Disagree

Strongly 

Disagree

a)

It is important for infants and toddlers to hold and play with 

different kinds of toys. 85% 12% 2% 0% 0%

b)

It is important for preschool children between the ages of 3 and 5 

years old to have books. 88 7 5 0 0

c) It is important for infants and toddlers to have books read to them. 93 7 0 0 0

d)

It is important to talk to infants and toddlers even if you don’t think 

they can understand what you are saying. 98 2 0 0 0

e)

It is important to talk with preschoolers about your/their daily 

events/activities. 90 10 0 0 0

f)

It is important that parents and caregivers teach preschoolers 

numbers and letters. 93 7 0 0 0

7. How much do you agree or disagree with each of the following statements: 

Statement

Strongly 

Agree Agree Neither Disagree

Strongly 

Disagree

a)

I believe it is important to maintain a quiet home environment in 

order for infants, toddlers, and preschoolers to learn. 29% 29% 27% 15% 0%

b)

I believe it is important to plan activities that are primarily “just for 

fun” for preschool children. 39 29 15 17 0

c)

I believe it is important for parents and/or caregivers to provide 

opportunities for preschoolers to select many of their own indoor 

activities.

44 42 12 2 0

d)

I believe it is important for parents and/or caregivers or providers 

to use treats, stickers, and/or stars to get toddlers to do activities 

that they don’t really want to do. 28 20 25 18 10

e)

I believe it is important to read stories to infants and toddlers every 

day. 88 12 0 0 0

f)

I believe it is important to let preschoolers watch television 

regularly without adult supervision. 5 0 7 39 49
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

15

8. How much do you agree or disagree with each of the following statements: 

Statement

Strongly 

Agree Agree Neither Disagree

Strongly 

Disagree

a)

I feel watching television is a good learning tool for infants and 

toddlers. 3% 21% 46% 28% 3%

b)

I feel flashcards are a good way to teach preschool children to 

read. 21 51 13 10 5

c) I feel toddlers learn to color best within pre-drawn forms. 8 34 40 8 11

d)

I feel it is important that preschoolers learn to play and work 

independently. 36 47 14 3 0

e)

I feel it is important that preschoolers see their 

parent(s)/caregivers read magazines, newspapers, and 

books.

67 26 8 0 0

f)

I feel it is important to provide toddlers with daily opportunities 

for developing social skills (i.e., cooperating, helping, talking) 

in the home.

80 20 0 0 0

9. How many children live in your home, including children of friends and/or relatives, in the following age groups?

a) None – there are no children under 18 years old in my home [PLEASE SKIP TO QUESTION 

14]

7%

b) 2 years old or under 83%

c) 3–5 years old 95

d) 6–10 years old 94

e) 11–13 years old 86

f) 14–17 years old 60

ACTIVITIES 

This next set of questions is about activities that you or someone in your home might do with young children. This would 

include parents, grandparents, aunts/uncles, mentors, or another caregiver. If you don’t have children in your home, or don’t 

care for children who are friends or relatives, please skip to question 14. 

10. How many times have you or someone else read to the child (children) in your home in the past week? 

a) Not at all 6%

b) Once or twice 11

c) Three or four times 25

d) Just about every day 58
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

16

11. How long does your child (children) enjoy being read to in one sitting? 

a) Doesn’t enjoy it 3%

b) Only a few minutes 22

c) 10–30 minutes 64

d) 45 minutes–one hour 11

e) More than one hour 0

12. In the past week, have you or someone in your family done any of the following things with your child 

(children)? [CHECK ALL OF THE ACTIVITIES YOU DID IN THE PAST WEEK] 

a) Told (him/her) a story 79%

b) Taught (him/her) letters, words, or numbers 71

c) Sang (him/her) songs or listened to music 74

d) Worked on arts and crafts with (him/her) 52

e) Played with toys or games indoors 74

f) Played a game, sport, or exercised together 62

g) Took (him/her) along while doing errands like going to the post office, the bank, or the store 71

h) Involved (him/her) in household chores like cooking, cleaning, setting the table, or caring for pets 76

i) Talked about what happened at school 69

j) Talked about TV programs or DVDs 36

k) Played counting games like singing songs with numbers or reading books with numbers 48

13. In the past month, have you or someone in your family done the following things with your child (children)? 

[CHECK ALL OF THE ACTIVITIES YOU DID IN THE PAST MONTH] 

a) Visited a library 50%

b) Gone to a movie 45

c) Gone to a play, concert, or other live show 17

d) Gone to a mall or store like Target 71

e) Visited an art gallery, museum, or historical site 19

f) Visited a playground, park, or gone on a picnic 76
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

17

g) Visited a zoo or aquarium 29

h) Talked about family history or ethnic heritage 41

i) Attended an event sponsored by a school, community, ethnic, or religious group 62

j) Attended an athletic or sporting event 33

k) Attended a church activity or church school 64

OTHER INFORMATION 

14. What is the highest level of education you have completed? 

a) Some high school 0%

b) High school graduate or GED 16

c) Vocational or trade school 0

d) Some college 32

e) College degree 53

15. This past year, what was your annual household income? 

a) Less than $10,000 14%

b) $10,000–$14,999 9

c) $15,000–$24,999 29

d) $25,000–$34,999 6

e) $35,000–$49,999 11

f) More than $50,000 31

16. What is your gender? 

a) Female 97%

b) Male 3
Starfish Inkster Family Literacy Movement Page of 19

Year Three Evaluation Report 

18

17. What is your racial or ethnic background? [IF HISPANIC “D”, ALSO ANSWER Q18] 

a) American Indian or Alaska Native 0%

b) Asian 2

c) Black or African American 65

d) Hispanic, Latino, or Spanish origin 3

e) White 16

f) Some other race 14