The Bridge Project Social 

Innovation Fund Annual 

Report

FINAL EVALUATION REPORT 

Submitted to the Bridge Project and Mile High United Way 

April 2017 
The Bridge Project Social 

Innovation Fund Annual Report

Final Evaluation Report

For more information, please contact: 

Sara Douglass Bayless, PhD 

sbayless@omni.org 

303-839-9422, extension 149

Melissa Richmond, PhD 

mrichmond@omni.org 

303-839-9422, extension 166 

For General Inquiries/Questions 

p. 303-839-9422 

f. 303-839-9420 

OMNI Institute 

899 Logan Street, Suite 600 

Denver, CO 80203 

www.omni.org 

Acknowledgements: This report is based upon work supported by the Social Innovation Fund (SIF), a 

key White House initiative and program of the Corporation for National and Community Service (CNCS). 

The Social Innovation Fund combines public and private resources to grow the impact of innovative, 

community-based solutions that have compelling evidence of improving the lives of people in lowincome communities throughout the United States. Program and evaluation activities described in this 

report were also made possible by Mile High United Way, the Bridge Project and the University of 

Denver. 
Prepared by OMNI Institute 

 Page i

Table of Contents

Executive Summary .......................................................................................................................................1

Key Findings ..................................................................................................................................................1

Introduction ..................................................................................................................................................1

Program Overview ....................................................................................................................................1

Elements of Early Literacy Instruction ......................................................................................................1

Evaluation Overview .....................................................................................................................................4

Evaluation Questions ................................................................................................................................4

Methods and Measures ............................................................................................................................5

Study Sample ............................................................................................................................................8

Implementation Study Results ....................................................................................................................10

School Year Program Dosage ..............................................................................................................10

Observations and Fidelity Reports ..........................................................................................................11

Read Well Overview ............................................................................................................................12

Tutoring Overview ..............................................................................................................................17

Impact Study Results...................................................................................................................................21

Research Questions ................................................................................................................................21

Method ...................................................................................................................................................22

Study Sample ..........................................................................................................................................22

Sample Characteristics ............................................................................................................................23

Baseline Equivalence Analysis.................................................................................................................24

Propensity Score Matching .....................................................................................................................26

Differential Attrition and Missing Data Analysis .....................................................................................29

Power Analysis ........................................................................................................................................31

Descriptive Outcomes .............................................................................................................................32

Multiple Regression Analyses .................................................................................................................34

Effects of Bridge Program on Outcomes.............................................................................................35

Linear Growth, Mixed-Effects Models ................................................................................................37

Deidentified Data ................................................................................................................................39

Summary: Confirmatory Questions ........................................................................................................41

Exploratory Questions ............................................................................................................................42

Summary: Exploratory Questions ...........................................................................................................46

Study Limitations ........................................................................................................................................47

Program Impacts .....................................................................................................................................47

Program Fidelity and Variation ...............................................................................................................47

Comparison Student Recruitment ......................................................................................................47

Missing Data ........................................................................................................................................48
 Prepared by OMNI Institute 

Page ii

Summary and Conclusions ..........................................................................................................................49

References ..................................................................................................................................................51

Appendix A – Read Well Observation Tool ................................................................................................. 52

Appendix B – Tutoring Observation Tool ....................................................................................................56

Appendix C – Descriptive Study Participant Data Tables ............................................................................59

Appendix D – Bridge Study Student Intake Forms ......................................................................................63

Appendix E – Comparison Student Intake Forms .......................................................................................66

Figures 

Figure 1. The Bridge Project Social Innovation Fund Early Literacy Program Model ...................................3

Figure 2. Bridge Dosage for Read Well, Tutoring, and GR8 Readers Programs Over Time ........................11

Figure 3. How Time Was Allocated in Read Well Sessions (N=16)* ...........................................................12

Figure 4. Literacy Skills Addressed in Read Well Sessions (N=16) .............................................................13

Figure 5. Educator Competencies ...............................................................................................................15

Figure 6. Effective Use of Rewards and Consequences ..............................................................................16

Figure 7. Average Length of Different Session Components: Variation by Site..........................................19

Figure 8. Percent of Tutoring Sessions that Explicitly Addressed Each Literacy Skill..................................20

Figure 9. Probability of Proficiency Over Time Among Propensity-Score Matched Bridge and Comparison 

Students ......................................................................................................................................................39

Tables 

Table 1 - Literacy Instruction Components ...................................................................................................2

Table 2 - Research Questions Addressed in the Current Study ....................................................................4

Table 3 - Study Data Sources ........................................................................................................................6

Table 4 - Study Population Enrollment and Bridge Attendance ...................................................................9

Table 5 - Bridge Program Dosage................................................................................................................10

Table 6 - Read Well Session Observation Information ...............................................................................12

Table 7 - Descriptive Statistics for Baseline Measures: Full Sample ...........................................................23

Table 8 - Descriptive Statistics at Baseline: Full Sample By Study Condition ..............................................25

Table 9 - Descriptive Statistics at Baseline: Matched Sample ....................................................................28

Table 10 - Proportions of Missing Data.......................................................................................................30

Table 11 - Baseline and Follow-up Outcomes: Full Sample ........................................................................32

Table 12 - Baseline and Follow-up Outcomes: Matched Sample ...............................................................33
 Prepared by OMNI Institute 

Page iii

Table 13 - Effects of Bridge Program on Outcomes: Full Sample ...............................................................36

Table 14 - Effects of Bridge Program on Outcomes: Matched Sample ......................................................36

Table 15 - Effects of Bridge Program on Outcomes: Mixed-Effects Models...............................................38

Table 16 - Descriptive Statistics By School at Baseline: Deidentified Sample ............................................40

Table 17 - Descriptive Statistics by Condition at Baseline: Deidentified Sample .......................................40

Table 18 - Effects of Bridge on Outcomes: Deidentified Sample ................................................................41

Table 19 - DRA-2 Proficiency Change Scores Among Bridge Students .......................................................43

Table 20 - Effects of Number of Bridge Services on DRA-2 Proficiency ......................................................44

Table 21 - Effects of Bridge Hours of Services Received on DRA-2 Proficiency ..........................................44

Table 22 - Within-Year Correlations Among Bridge Program Activity Dosage ...........................................45

Table 23 - Effects of Individual Bridge Programs on DRA-2 Proficiency .....................................................45

Table 24 - Long-Term DRA-2 Proficiency Change Scores ............................................................................46
 Prepared by OMNI Institute 

 Page 1 

Executive Summary

The Bridge Project promotes education as the key to self-sufficiency and breaking the cycle of poverty. 

Specifically, the mission of the Bridge Project is to provide a path for youth in Denver’s public housing 

neighborhoods to graduate from high school and go on to college or a vocation by engaging them in 

educational opportunities and facilitating the development of life skills and self-sufficiency. The Bridge 

Project provides a comprehensive early literacy intervention targeting students in K-3rd grade, with the 

goal of achieving reading at grade level by the end of third grade. Through a combination of three direct 

literacy programs, including small group instruction (Read Well), one-to-one tutoring, and the provision 

of books to students (GR8 Readers), the Bridge Project offers participants the opportunity to experience 

many facets of literacy and allows them to take an active role in their learning. Through expanded 

resources and the support of the Early Literacy Social Innovation Fund, the Bridge Project anticipated a 

significantly greater increase in the percentage of students reading at grade-level or higher among 

students living in public housing with access to the Bridge program than among students living in public 

housing without access to Bridge services. 

To determine the effectiveness of this intervention, OMNI Institute (OMNI) developed a mixed-methods 

evaluation. This evaluation consisted of a quasi-experimental, between-group design to assess whether 

students who receive Bridge Project services show greater gains in literacy skills than similar students 

who did not receive Bridge services. Students in K-3rd grades entering the Bridge Project in four public 

housing neighborhoods served as the treatment group. Comparison students were recruited from two 

similar public housing communities in which Bridge Project services are not currently available. Reading 

and services data were collected and analyzed to make comparisons between the two groups. Full 

descriptions of the program, methods, measures, and findings are presented in this report. Key findings 

are highlighted below: 

Key Findings

• A total of 371 students who started in 

grades K-3 received Bridge Project 

literacy services over four years. 

• During the 2015-16 academic year, 

Bridge students received an average of 

37 Read Well sessions, 16 tutoring 

sessions, and five GR8 Readers books 

per student. This reflects an increase in 

program dosage from previous years. 
 Prepared by OMNI Institute 

Page 2

• Observations conducted during the 2015-16 academic year indicated that the Read Well and 

tutoring sessions were largely on-task and provided high-quality instruction to Bridge students. High 

quality indicators included demonstrated student learning in the five fundamental literacy skill areas 

(comprehension, fluency, vocabulary, phonics, and phonemic awareness) and educator 

competencies, including provision of clear instructions, attentiveness to student needs, and 

individualized student instruction. 

• The Bridge program appears to 

significantly benefit students’ reading 

proficiency, particularly after the first 

year of programming. Analyses of the

quasi-experimental, between-group 

design find that students who receive 

Bridge Project services showed greater 

gains in literacy skills than similar 

children who did not receive Bridge 

services. By their second and third 

years in the program, Bridge students 

were outperforming their peers. 

 

 

• Among Bridge students, the number of programming elements received (i.e., Read Well, tutoring, 

GR8 Readers, and other services) and the amount of programming received in each element was 

positively related to reading proficiency. Specifically, it appears that the number of GR8 Readers 

books that Bridge students receive was positively related to reading proficiency above and beyond 

the effects of Read Well, tutoring, and other services provided by the Bridge Project. However, it is 

important to note that GR8 Readers books are incorporated into other aspects of the Bridge 

Project’s literacy programming, including Read Well and tutoring, and therefore should not be 

considered a unique effect. 

• Overall, it does not appear that the Bridge Project literacy program is related to school attendance. 

For this secondary outcome, there is a consistent lack of differences among Bridge and comparison 

students across analytic models. Analyses may have been limited by the lack of variability in 

attendance, as both Bridge and comparison students had consistently high attendance rates over 

the course of the study. In practice, the Bridge program may not be able to impact attendance rates 

among a sample that is generally attending school regularly. 
 Prepared by OMNI Institute 

 Page 1 

Introduction

Program Overview 

The Bridge Project promotes education as the key to self-sufficiency and breaking the cycle of poverty. 

Specifically, the mission of the Bridge Project is to provide a path for youth in Denver’s public housing 

neighborhoods to graduate from high school and go on to college or a vocation by engaging them in 

educational opportunities and facilitating the development of life skills and self-sufficiency. The Bridge 

Project provides children and youth in Denver’s public housing neighborhoods with services that will 

lead to improved academic skills, graduation from high school, and opportunities to attend college. The 

Bridge Project offers a number of programs designed to help facilitate the development of academic 

success including homework help, mentoring, access to computers, social skills training, and support 

from positive and healthy adult and peer relationships. 

In addition to these services, the Bridge Project offers a comprehensive literacy intervention targeting 

children in Kindergarten through third grade, with the goal of achieving reading at grade level by the end 

of third grade and improving subsequent academic achievement. The Bridge Project literacy 

intervention includes three program components: 

Read Well is an evidence-based reading intervention program, delivered to small groups of students. 

Read Well features mastery-based and research-validated instructional strategies, unique sound 

sequence, differentiated instruction with flexible pacing, and ongoing assessment/progress monitoring. 

Trained educators provide this service to children at each of the four sites throughout the program year. 

During the academic year, Read Well is delivered during 45-minute sessions three times per week; 

during the summer, Read Well is delivered in 30 minute sessions four times per week. 

One-to-one tutoring is an intervention in which children are matched with trained adult tutors. Once 

matched, students meet with the same tutor at least once a week for a 45-minute tutoring session. 

During this session, the children and tutors focus on reading strategies and building literacy confidence. 

Tutoring is offered throughout the academic year, but not during summer months. 

GR8 (Great) Readers is an intervention designed to help children build a home library and increase 

reading skills and enjoyment through access to age and content appropriate books. GR8 Readers 

provides K-3 students with eight books over the course of the school year and eight books over the 

summer. Bridge educators develop the menu of books available and students are then able to select the 

books that they are most interested in reading and owning. During the academic year, reading of GR8 

Readers books is integrated into one-to-one tutoring sessions and students are typically allowed to take 

each book home to keep after 4-5 tutoring sessions; during the summer, students select two sets of four 

books to take home twice during the summer.

Elements of Early Literacy Instruction 

This three-pronged intervention aims to address five core elements of early literacy instruction, as 

described by the National Institute of Child Health and Human Development and outlined in Table 1. 
 Prepared by OMNI Institute 

Page 2

TABLE 1 - LITERACY INSTRUCTION COMPONENTS 

Element Definition

Phonemic Awareness The ability to hear, identify, and manipulate the individual sounds (phonemes) in spoken words. 

Phonics Understanding the relationship between the letters (graphemes) of written language and the 

sounds (phonemes) of spoken language; learning letter-sound relationships. 

Fluency The ability to read with accuracy, and with an appropriate rate, expression, and phrasing. 

Vocabulary Understanding the meaning of the words we must know to communicate effectively in listening, 

speaking, reading, and writing. 

Comprehension The ability to understand, remember, and communicate with others about what is read. 

A program logic model depicting the Bridge early literacy intervention components appears on the 

following page (see Figure 1).
Prepared by OMNI Institute 

 Page 3 

Figure 1. The Bridge Project Social Innovation Fund Early Literacy Program Model 

 

 

 
Prepared by OMNI Institute 

Page 4 

Evaluation Overview

The evaluation of Bridge Project literacy programming consists of a mixed-methods approach, including 

a quasi-experimental, between-group design to assess whether children who receive Bridge Project 

services show greater gains in literacy skills than similar children who do not receive Bridge services. 

Children in K-3rd grades entering the Bridge Project in four public housing neighborhoods serve as the 

treatment group. Because all children in these neighborhoods are eligible for and can receive Bridge 

services, comparison children were recruited from two similar public housing neighborhoods in which 

Bridge Project services were not available. The comparison group thus provides the counterfactual 

condition for the study, demonstrating literacy outcomes for students similar to those served by Bridge 

who did not receive the intervention. For students whose parents consent to participating in the study, 

data on family and child demographic characteristics (e.g., race, ethnicity, age, parental expectations of 

children’s academic success, housing community) are collected at intake and include children’s Denver 

Public Schools (DPS) identifiers. This allows the project to access school literacy assessment data for all 

DPS students who consent to participate in the study.1 Analyses over the course of the study compare 

reading assessment over time for children in the treatment and comparison groups to quantify program 

impact. Propensity score matching techniques are used to adjust for pre-intervention differences 

between students in the Bridge (treatment) and comparison groups. 

Evaluation Questions 

The Social Innovation Fund study of the Bridge Project early literacy program seeks to address both 

impact and implementation questions, as outlined in the project’s approved Sub-Grantee Evaluation 

Plan. Research questions addressed in the current report appear in Table 2. 

TABLE 2 - RESEARCH QUESTIONS ADDRESSED IN THE CURRENT STUDY 

Implementation Questions 

1. What level of program intervention was provided? 

a) How many Bridge children were matched to a tutor? 

b) How many sessions of Read Well® instruction were provided to participants? 

c) How many hours of one-on-one instruction were provided to participants? 

d) How many books were provided to students as part of GR8 readers? 

e) Did these differ as a function of housing community, grade or other demographic characteristics? 

f) Did these differ during the academic year and over the summer? 

2. To what extent did volunteer tutors and education specialists deliver the program with fidelity? Was this 

consistent across Bridge sites? 

 

1

 Most, but not all, students served by the Bridge Project attend Denver Public Schools. Outcome data are not 

available for study participants attending a non-DPS school. 
 Prepared by OMNI Institute 

Page 5

Implementation Questions 

3. What are implementation challenges associated with each type of intervention, do these differ as a function 

of child or site characteristics, timing of the intervention, and what are acceptable solutions to these 

challenges (i.e., that do not violate fidelity to the model)? 

Impact Questions 

Confirmatory Impact Questions: 

1. Do more Bridge children show greater improvements in reading at or above grade level than comparison 

children at the end of each year and at the end of the study (binary DRA-2)? 

2. Do Bridge children show greater reading gains than comparison children, within each academic year and 

across all years of the study (truncated DRA-2)?

2

 

3. Do Bridge children show greater improvement in rates of school attendance than comparison children, 

within each academic year and across all years of the study (secondary outcome)?

Exploratory Impact Questions: 

5. Are more Bridge children reading at or above grade level by the end of third grade than comparison children? 

6. Does the amount or type of literacy programming predict reading gains? For example, does the number of 

years of participation in the program predict reading gains? 

7. What is the strength of the relationship between number of hours of participation and reading gains? Does 

this differ across program component? 

8. Which literacy program activities are most closely associated to reading gains; what are the critical program 

elements for success? 

9. Are results for grades K-3 maintained for Bridge children who reach fourth or fifth grade during the study 

period? Do these Bridge children show greater long-term gains in reading achievement than comparison 

children? 

Methods and Measures

Data for this mixed methods study were collected in a variety of methods. Quantitative measures 

include: a) an information sheet completed by parents of study participants; b) services data as entered 

by staff responsible for delivering Bridge Project programming; and c) reading and attendance data from 

Denver Public Schools (DPS) Department of Assessment, Research and Evaluation (ARE). Qualitative data 

were collected through observations of both Read Well and tutoring sessions. Key data sources 

reflected in this report are outlined in Table 3. 

 

2

 For question 1b, we revised the analysis strategy to explore grade level equivalencies rather than conducting a 

truncated grade level analysis. We learned that teachers administer the full DRA-2 independent level in the spring, 

therefore, this truncated approach was no longer needed and was replaced with a measure of grade level 

discrepancy. 
 Prepared by OMNI Institute 

Page 6

TABLE 3 - STUDY DATA SOURCES 

Source Name Description 

Study Intake 

Forms 

Parents of study participants, both Bridge Project and comparison students, complete a 

detailed intake form upon enrollment in the study.3 Data collected from these forms 

include information about the family and the child, including: number of family 

members, number of children, household income, parent education, language spoken at 

home and child characteristics such as age, gender, race, grade, preschool attendance. 

The forms also include information about the home environment, such as the frequency 

of family dinners, number of books owned by the child, frequency of reading with the 

child and parental expectations about school for the child. 

Services Data Bridge site staff use a web-based data system to document student attendance during 

Read Well and Tutoring sessions. Specifically, student attendance data are documented 

by site administrative staff and entered into the database. The number of GR8 readers 

books given to a youth is also entered into this database. This log provides a cumulative 

list of individual students participating in GR8 Readers, the title(s) of the book(s) they 

received, and the date(s) on which they were given each GR8 Readers book to take 

home. OMNI staff extracts the services data from the Bridge database for analysis. 

Observations During each year of the study, members of the research team conduct structured 

observations of Read Well and tutoring sessions delivered at each of the four Bridge 

sites. Over the past five years, these observations occurred during the academic year and 

were conducted using structured observation protocols designed for each program 

component (i.e., one for Read Well and one for tutoring) to ensure consistent data 

collection practices. The use of this observation tool has allowed for the standardized 

collection of quantitative data in several key areas to examine consistency across 

sessions. Qualitative notes were also taken to provide contextual and descriptive 

information. The full observation tools can be found in Appendices A and B. 

Observations were conducted during randomly selected sessions (for Read Well) or 

program weeks (for tutoring) from among the full list of days on which Bridge programs 

were open during the school year. Selected observation dates were provided to staff at 

each of the Bridge sites in advance to confirm that the respective programming was 

scheduled to occur on the selected dates. 

Quantitative data gathered from each observation tool were aggregated and analyzed 

using SPSS. Qualitative observation notes were analyzed to supplement quantitative 

findings where appropriate. Data gathered during each observation was documented 

using the observation protocol and analyzed to describe the components of Read Well 

and tutoring sessions that appeared to influence session quality, content, and 

effectiveness. 

 

3

 Parents of Bridge Project participants also complete an intake form at the start of each Bridge enrollment period 

(i.e., each academic year and each summer). 
 Prepared by OMNI Institute 

Page 7

Source Name Description 

School-based 

Data 

Each year, the research team submits a list of all students enrolled in the study to be 

matched to literacy assessment data collected by DPS. In November 2016, DPS ARE 

provided data for students enrolled in the study and in DPS. Specifically, the project 

received the following data from DPS: 

DRA-2 reading scores. The DRA-2 is a standardized reading assessment that evaluates 

reading on different domains including word analysis, oral fluency and comprehension 

(Pearson Education, 2011). The DRA-2 has been demonstrated to be a reliable reading 

assessment and has concurrent validity with other reading assessments, including the 

Grays Oral Reading Test (GORT-4), the DIEBELS, and with teacher ratings (Rathvon, 

2008). Each spring, teachers in DPS administer the DRA-2 with K-3 students to assess 

students’ independent reading level. Benchmark assessments are leveled from A to 40. 

The following benchmarks were identified by the Colorado Department of Education to 

indicate grade-level reading at the end of each of grade: Kindergarten = 3; 1st grade = 18 

2

nd grade = 28; 3rd grade = 38. Based on these benchmarks, the teachers assign each 

student an independent reading level which identifies the student as reading below 

grade level, on grade level or above grade level. Additionally, the DRA-2 technical 

manual indicates grade-level cut points for students who are proficient/independent 

(reading at or above grade-level); instructional level (a ‘yellow flag’), needing to be 

closely monitored and possibly in need of additional support; and intervention level (a 

‘red flag’) or in need of significant intervention. With these categories as a reference 

point, students in the Bridge Project study are classified into three categories based on 

their reading level: 

▪ Proficient/Independent: At or above grade level; 

▪ Partially Proficient: Below grade-level, but not below the ‘red flag’ level; and 

▪ Not proficient, scoring in need of more intensive support, ‘red flag’. 

DRA-2 Independent Reading Level assigned by teachers during the prior spring serves as 

the baseline measure for all students who enrolled in the study in 1st, 2nd or 3rd grades. 

The baseline level reflects the student’s performance on the DRA-2 during the spring of 

the year prior to study enrollment (e.g., students enrolled in the study as 1st graders have 

baseline DRA-2 Independent Reading Level data from the spring of their kindergarten 

year). Independent Reading Level assigned by teachers during the following spring 

serves as the posttest measure Each subsequent spring, the project requests DRA-2 data 

for students who are enrolled in the study. 

DRA-2 Word Analysis Mastery. Seven of the DRA-2 Word Analysis Tasks, administered at 

start of the academic year, serve as the baseline measure for all students enrolled in the 

study during their kindergarten year. The seven tasks are: 

▪ Task 1: Identify a picture whose name rhymes with the name of the first picture in 

the set 

▪ Task 2: Identify a picture whose name begins with the same sound as the name of 

the first picture in the set 
 Prepared by OMNI Institute 

Page 8

Source Name Description 

▪ Task 3: Isolate the initial sound of a word given by the teacher 

▪ Task 4: Demonstrate that the student understands language used to talk about 

printed language concepts using their first and last names 

▪ Task 5: Name randomly placed uppercase letters 

▪ Task 6: Name randomly placed lowercase letters 

▪ Task 7: Demonstrate that the student understands language used to talk about 

printed language concepts using words in sentences 

Students are assigned a score of between zero and two for each task, reflecting “no or 

little control” (0), “some control” (1) and “gaining or having control” (2). These data 

serve as the baseline measure for all kindergarten students enrolled in the study, as the 

full DRA-2 assessment is not administered with kindergarten students until the spring of 

the kindergarten year. Based on guidance received from DPS, study students with 

mastery levels of “gaining or having control” on four or more of the seven tasks are 

defined as proficient. 

Attendance. Denver Public Schools tracks student attendance in minutes, and the annual 

attendance rate is calculated as follows: total number of minutes a child attended a DPS 

school/total number of minutes a child was enrolled in DPS. 

Study Sample 

The treatment and comparison groups in the study are divided as follows: 

Treatment Group: Children in grades K-3 entered the Bridge Project in four public housing 

neighborhoods (Columbine, Quigg Newton, South Lincoln/Lincoln Park, and Westwood). Specifically, all 

kindergarten students participating in the Bridge project, and 1st-3rd grade students who had never 

participated in Bridge programming were eligible to enroll in the treatment group. 

Comparison Group: Because all children residing in public housing neighborhoods served by the Bridge

Project were eligible to receive Bridge services, comparison children were recruited from two similar 

public housing communities, Westridge and Sun Valley, in which Bridge Project services were not 

available. The comparison group thus provides the counterfactual condition for the study. 

A total of 543 students (389 Bridge; 154 comparison) were enrolled into the study. Table 4 provides the 

number of students who enrolled into the study by condition and year of enrollment. Comparison 

students enrolled into the study during the academic year. Bridge participants enrolled into the study 

either at the beginning of the academic year or at the beginning of the summer. Bridge students who 

enrolled during the summer had to continue enrollment into the following academic year to be included 

in the study (i.e. students receiving ‘summer-only’ programming were not included in the evaluation and 

are not reported in Table 4). 
 Prepared by OMNI Institute 

Page 9

TABLE 4 - STUDY POPULATION ENROLLMENT AND BRIDGE ATTENDANCE 

Count of students by condition and enrollment period 

Study Group 2012-13 Summer 

2013

2013-14 Summer 

2014

2014-15 Summer 

2015 

2015-16 Total 

Comparison 

Students

19 0 33 0 67 0 35 154 

Bridge Students 95 40 72 29 82 35 36 389 

Total count of students served by Bridge by time period 

Study Group 2012-13 Summer 

2013

2013-14 Summer 

2014

2014-15 Summer 

2015 

2015-16

Bridge Students 95 69 128 91 166 117 116 

Total count of students served in each grade by academic year 

Grade 2012-13 2013-14 2014-15 2015-16

Kindergarten 45 35 44 16

1st Grade 18 41 36 22

2nd Grade 14 19 45 32

3rd Grade 17 28 24 24

4th Grade - 5 14 13

5th Grade - 4 6 

6th Grade - 3 

Total count of students served at each site by academic year 

Bridge Site 2012-13 2013-14 2014-15 2015-16

Columbine 12 12 24 26

Quigg Newton 23 37 22 23

South Lincoln 14 13 47 27

Westwood 45 66 74 40
 Prepared by OMNI Institute 

Page 10

Implementation Study Results

Implementation questions focus on early literacy program dosage received by students served; variation 

in program dosage by Bridge site, student characteristics, and programming period (school year or 

summer); and implementation fidelity. This report section first explores the level and variation of 

program intervention for Bridge students, followed by a discussion of qualitative data gathered to assess 

the degree of fidelity to the program model overall and across sites. Implementation challenges are also 

summarized in the Key Lessons Learned and Next Steps section at the end of this report. 

What level of program intervention was provided? Did the level of programming 

intervention vary by Bridge Project site, student characteristics or programming 

period? 

School Year Program Dosage 

Service dosage data were available for 126 of the Bridge study students served by the program during 

the 2015-16 school year. As shown in Table 5, 81 students received at least one Read Well session, with 

an average of about 37 sessions per student; 83 students received at least one tutoring session, with an 

average of about 16 sessions per student; and 80 students received at least one GR8 Readers book, with 

an average of five books per student. Across these programs, there was an increase in service dosage 

from the 2014-15 year, such that Bridge students in 2015-16 received an average of eight more Read 

Well sessions, three more tutoring sessions, and nearly two more books than Bridge students in 2014-

15. 

TABLE 5 - BRIDGE PROGRAM DOSAGE

Activity Type Count (N) Min. Max. Average 

2015-16 Academic Year 

Matched to Service Data File 126 n/a n/a n/a 

Read Well Sessions 81 1 82 36.53 

Tutoring Sessions 83 1 35 15.73

GR8 Readers Books Received 80 1 15 5.09

2014-15 Academic Year 

Matched to Service Data File 167 n/a n/a n/a 

Read Well Sessions 130 1 82 28.25 

Tutoring Sessions 103 1 37 12.8 

GR8 Readers Books Received 58 1 7 3.31 

2013-14 Academic Year 

Matched to Service Data File 104 n/a n/a n/a 

Read Well Sessions 91 1 85 27.04 

Tutoring Sessions 62 1 49 14.82 

GR8 Readers Books Received 38 1 8 2.82 

2012-13 Academic Year

Matched to Service Data File 84 n/a n/a n/a

Read Well Sessions 72 1 57 14.93
 Prepared by OMNI Institute 

Page 11

Tutoring Sessions 36 1 30 11.69

GR8 Readers Books Received 18 1 13 5.11

The average number of Read Well and Tutoring sessions generally increased over four years of the 

Bridge study (i.e., from 2012-13 to 2015-16; see Figure 2). While the average number of GR8 Readers 

books received by students in 2015-16 was similar to the average number of books received by students 

in 2012-13, this average for 2015-16 represented an increase from 2013-14 and 2014-15. Additionally, 

the total number of students who received books was over four times higher in 2015-16 than in 2012-

13. 

Figure 2. Bridge Dosage for Read Well, Tutoring, and GR8 Readers Programs Over Time 

Observations and Fidelity Reports 

To what extent did Bridge Project staff and volunteers deliver the program with 

fidelity? Was this consistent across Bridge Project sites? What are implementation 

challenges associated with each program component? 

In the 2015-2016 academic year, one OMNI staff member conducted a total of 16 discrete observations 

of Read Well sessions across the four Bridge sites. Specifically, a total of four observations were 

conducted per site, including two observations of the K/1 Read Well group and two observations of the 

2/3 Read Well group at each site. In addition, there were 12 tutoring observations, three in each of the 

four Bridge sites. 

15

12

5

27

15

3

28

13

3

37

16

5

0

5

10

15

20

25

30

35

40

ReadWell Tutoring Sessions GR8 Readers Books

Average

Number of Sessions/Books

2012-13 2013-14 2014-15
 Prepared by OMNI Institute 

Page 12

Read Well Overview 

The observed Read Well sessions were led by a trained Educator, included an average of seven students 

per session, and were typically 42.8 minutes in duration. Table 6 summarizes general information about 

the observed sessions. 

TABLE 6 - READ WELL SESSION OBSERVATION INFORMATION 

N Mean Minimum Maximum 

Duration (minutes) 16 42.8 36.0 49.0 

Total Number of Students 16 7.0 4.0 11.0 

Number of Boys 16 3.5 1.0 7.0 

Number of Girls 16 3.5 1.0 6.0 

The time spent in Read Well instruction included direct instruction, literacy activities and some activities 

not focused directly on literacy, such as games and art activities offered as a reward; and opportunities 

for students to rest through physical movement or time spent outside. As seen in Figure 3, most session 

time was devoted to Read Well instruction. 

 Figure 3. How Time Was Allocated in Read Well Sessions (N=16)*

*Outer bounds represent the minimum and maximum number of minutes spent in a session, grey boxes represent 

the middle 50% of sessions, box dividers represent the median number of minutes, and diamonds represent the 

average number of minutes. 

Similar to the findings from 2014-15, students are spending the clear majority of session time on Read 

Well curriculum under the supervision of the Educator. We did observe a slight increase in the average 
 Prepared by OMNI Institute 

Page 13

amount of time spent on Read Well from 29 minutes in 2014-15 to 32 minutes in 2015-16. Additionally, 

the average amount of time spent on non-literacy activities increased from eight minutes to 10 minutes. 

During 2014-15, much of the non-literacy time was spent on regaining student focus after transitions or 

taking time to address student behavioral issues, whereas during the current year, this time also 

included the newly instituted Warm Welcome and Reflection portions of each session. 

During the 2015-16 year, the Bridge Project instituted two new components to the Read Well session, 

the Warm Welcome and the Reflection. The Warm Welcome was a few minutes at the beginning of each 

session for the Educator to discuss with the students what the session will look like, expectations, and to 

check in with student about how they were doing. The Reflection portion took place at the end of each 

session and allowed the Educator to review with the students what they covered, and the students to 

share how they felt the session went. Of the 16 sessions that were observed, 10 included time for Warm 

Welcome, and 12 involved Reflection. The Warm Welcome and Reflection components were instituted 

as part of Bridge’s participation in the Youth Program Quality Intervention (YPQI) process. The YPQI 

resulted from learnings from the prior 2014-15 observations, during which behavior management 

concerns were identified, and was intended to help staff identify elements that could increase the 

quality of programming; Warm Welcome and Reflection were added to increase engagement to

promote more positive classroom behaviors. 

Read Well Activities and Lesson Components: The Read Well curriculum includes four different types 

of activities: Decoding Activities, Story Reading, Partner Reading, and Follow-Up Activities. Observed 

sessions usually began with Decoding Activities, which are based on the Decoding Sheets provided by 

the Read Well curriculum. All four sites had the students work through the Decoding Sheets verbally as a 

group. Decoding Activities were generally followed by Story Reading, Follow-Up Activities, and Other 

Activities when included. Other Activities were mostly literacy-focused, except for a few that were nonliteracy focused. These activities included drawing/coloring, games, and practicing handwriting skills. 

Literacy Skill Areas: 

Observations assessed the extent to which 

each of the five fundamental literacy skill 

areas were explicitly addressed for each grade 

level. In comparison to observations conducted 

in 2014-15, there was a noticeable increase in 

the number of sessions that addressed 

vocabulary with grades K-1, from two to nine. 

Comprehension, phonics, phonemic 

awareness, and fluency were addressed in the 

clear majority (93.8%-100%) of sessions. 

Fluency was addressed in 100% for both levels, 

which is an increase from the 43.7% of sessions 

that covering the skill area in 2014-15. 

Figure 4. Literacy Skills Addressed in Read Well 

Sessions (N=16) 

 
 Prepared by OMNI Institute 

Page 14

Additionally, there was an increase in the number of sessions that involved vocabulary from eight to 14. 

Use of Small Groups: Bridge educators occasionally divide students into small groups during Read Well 

sessions. Specifically, in 2015-16, students were divided into small groups in half (eight) of the observed 

sessions, which was an increase of three sessions from 2014-15. The use of small groups was observed 

at all but one Bridge site. Most commonly, students were separated into groups by grade or reading 

level. During four of the eight sessions, one educator was responsible for supervising multiple small 

groups. During these sessions, the groups that did not have the undivided attention of the educator 

were at times distracted, but generally kept each other focused and on task. In the other four sessions, 

each small group was supervised by either the educator or another adult and fewer distractions were 

observed. 

Read Well Session Environment: All observed Read Well sessions were held in a designated space that 

was separated from other Bridge activities. The space included a place to store Read Well materials, and 

classrooms at all sites had a whiteboard at the front of the room to which students directed their 

attention. Some observed sessions had students sitting at one large table or at tables arranged in a 

semicircle, and others had them sitting at multiple tables. Although having all students at one large table 

or in a semicircle seemed to make it easier for the students to see and track one another while they read 

aloud or answered questions, having multiple tables allowed Educators to easily divide students into 

groups for an activity if they so desired. 

Environmental Distractions: The observer rated each session as having many, some, or no external and 

internal distractions. Internal distractions included those that were internal to the small group (e.g., 

talking or misbehaving by other students, objects brought into the classroom by students, or students 

leaving the room). External distractions included those that were external to the small group (e.g., 

looking out of a window, or other students or adults entering the room). In nine of the 16 observed 

sessions, at least one student entered the session late, was sent out temporarily, or left the session 

early. This was a decrease from 2014-15 in which all sessions were observed to have students coming 

and going during Read Well sessions. Students were observed to be disruptive in 11 of the 28 (39.2%) 

instances when they entered late, left early, or went out temporarily. This was a significant 

improvement from the previous round of observations in which almost 80% of such instances were 

disruptive. Adults other than the Educator entered the classroom in three of the 16 observed sessions, 

with the number of other adults entering during a session never being over two. Additionally, this 

occurred at just one site, and reasons for other adults entering included taking attendance and 

removing students who were going home or needed additional discipline. The observer also rated each 

instance of other adults entering the room on whether it was disruptive to the session. There was only 

one instance of another adult entering the session that was distracting to the students. This marks a 

decrease from last year’s findings, in which other adults who entered Read Well sessions caused 

distractions on numerous occasions. 

Student Learning and Behaviors: The observer rated student behaviors during the Read Well sessions. 

Overall, students were observed to both enjoy and benefit from the Read Well curriculum. When 
 Prepared by OMNI Institute 

Page 15

students were focused on curriculum, they were generally engaged with the material and were eager to 

help the Educator with tasks such as cleaning up. Students also displayed behavioral issues less 

frequently this year than in the past. The number of sessions that were observed in the 2015-16 

academic year to have “many” internal distractions was 2, which was an improvement from 2014-15 in 

which 10 of the 16 sessions had many distractions. During both years of observations, internal 

distractions were comprised of instances involving students misbehaving. Out of the 113 students 

observed, 23 (20.3%) displayed behavioral issues. This marks an improvement from the previous round 

in which 55.8% of students observed displayed behavioral issues. All but four of the 16 observed 

sessions had at least one student disrupt the class during the session, with an average of 1.4 students 

acting out per session. Half of the sessions in which there were student disruptions involved only one 

student. Observed behavioral issues included talking while the Educator was teaching, distracting other 

students, refusing to follow instructions, and standing up and moving around the classroom at 

inappropriate times. 

Instructional Skill of Educators: Overall, Educators demonstrated strong abilities in delivering the Read 

Well Curriculum. In all observed sessions, lessons and materials were planned and content and materials 

were generally appropriate for students’ academic level. As shown in Figure 5, Educators often provided 

clear instructions, provided encouragement, and were attentive to the needs of their students in most 

observed sessions. 

Figure 5. Educator Competencies 

1 2

3

3

2

13

10

14

Attentive to Needs of Students

Provides Encouragement

Provides Clear Instructions

Number of Sessions

Never Rarely Sometimes Mostly Always N/A

*N/A indicates that rewards/consequences were either not used or were unnecessary during the session. 

Effective Use of Rewards and Consequences: The observer also documented the efficacy of rewards 

and consequences used by Educators. As seen in Figure 6, rewards were used in 11 of the 16 observed 
 Prepared by OMNI Institute 

Page 16

sessions and were perceived to be effective. The number of sessions where the use of rewards was 

“always” effective increased from two during the 2014-15 academic year to six in 2015-16. Examples of 

instances where the use of a reward was less effective includes its failure to be used as an incentive for 

improved behavior, and students not noticing when an educator placed a PBIS ticket by them and 

therefore were not knowledgeable of why they received it. All Bridge sites used a certain degree of 

student choice in how to use “free” time as reward for positive behavior and performance. Verbal praise 

was observed at all sites, and two sites utilized the distribution of Positive Behavior Interventions and 

Supports (PBIS) tickets. It was observed that verbal praise that was specific to both the behavior and 

individual in combination with a PBIS ticket appeared to be most effective. 

Consequences were used in nine of the 16 sessions, which was a decrease from 2014-15 in which 12 of 

the 16 sessions required consequences. Additionally, the number of sessions where discipline was 

“always” used effectively increased from one to four. Disincentives that were used included verbal 

reprimands directed at individuals or the entire group, asking students to leave the group and sit alone, 

and taking away group rewards such as a promised free time. Overall, disincentives that were directed 

at individuals seemed more effective than those directed at the entire group. At one site, the educator 

would ask for help from the entire class when an individual was either acting out or distracting others. 

The educator would ask the students to raise their hand and offer suggestions for how the group can 

help the individual re-engage or become more focused. This strategy appeared to be effective in keeping 

the entire group on track. 

Figure 6. Effective Use of Rewards and Consequences 

Bridge Educator Observations 

To expand opportunities for peer-learning and information sharing between Bridge staff at all sites, as 

well as continue to monitor program fidelity, each Read Well Educator performed two observations of 

sessions that were conducted by another Educator. During the 2015-16 school year, eight educator 
 Prepared by OMNI Institute 

Page 17

observations were performed, two at each site. In general, the data collected during the Educator 

observations was similar to data collected by OMNI. Specifically, for both OMNI and Educator observed 

sessions, each Read Well session was well planned, appropriate in content, and most students were 

engaged with the material. 

Additionally, areas of difference from 2014-15 such as quality of transitions, student behaviors, and 

educator competencies were more closely aligned during the current round of observations. Despite the 

many similarities, it is important to note that the opportunity each Educator had to observe one 

another’s methods offered a wealth of knowledge, and at times, remaining solely focused on filling out 

the data collection tool was not a priority. For this reason, there were several observation areas that 

were not fully completed by the Educators, making a full comparison difficult. 

Read Well Implementation Strengths and Challenges 

Bridge Project students were observed to both enjoy and benefit from Read Well programming. Overall, 

students were engaged with the material, followed Educators’ instructions, and demonstrated learning 

of literacy skills during Read Well sessions. Observation data also revealed that overall Educators 

demonstrate strong abilities in delivering the Read Well curriculum by providing clear instructions, 

encouraging students, and attending to the needs of students. 

Compared to last year, this round of observation findings showed an increased amount of time students 

spend working on Read Well curriculum. the 2015-16 academic year, the average time each session 

utilized Read Well material was 32 minutes, which was an increase from the 29-minute average in the 

previous year. Additionally, the number of students coming and going from the session, which was a 

cause for much distraction during the 2014-15 academic year, decreased during the 2015-16 year. 

Effective use of rewards and consequences has been an area of growth for the Read Well program over 

the years, although developments in this area continue. As a result of the YPQI process, educators are 

trying new approaches to address student behavior and effective encouragement and incentive, which 

were noted as attributing to the improvements in this area. 

Tutoring Overview 

As a component of the Bridge Project evaluation, OMNI Institute research staff conducted observations 

of tutoring sessions across the four Bridge implementation sites. There were three primary goals for the 

observations: 

• Enable the research team to provide program staff with a systematic description of tutoring 

sessions across all four sites; 

• Provide data on the extent to which tutoring sessions are implemented with fidelity to the 

instructional model outlined by the Bridge Project; and 

• Provide recommendations to promote the ongoing improvement of tutoring programming, as 

necessary 
 Prepared by OMNI Institute 

Page 18

Between January and May 2016 two research team members observed three tutoring sessions in each 

of the four Bridge sites for a total of twelve observations. The research team randomly selected the 

week during which tutoring observations would occur and then worked with site staff to identify K-3 

tutor pairs scheduled to meet during the selected week. 

Consistent with the Bridge tutoring guidelines, most tutoring sessions followed this general agenda: 

1. Homework time (observed in six out of 12 (50%) sessions) 

2. GR8 Readers book reading (observed in 10 out of 12 (83.3%) sessions) 

3. GR8 Readers packet work (observed in four out of 12 (33.3%) sessions) 

4. Skill-building activity (observed in four out of 12 (33.3%) session 

5. Reward time (observed in 10 out of 12 (83.3%) sessions) 

Allocation of Session Time: The amount of time spent on literacy-focused activities varied among the 

tutoring sessions observed. Sessions lasted an average of 44 minutes, with a minimum session length of 

25 minutes and a maximum session length of 58 minutes. All tutoring sessions included time for literacyfocused activities, with an average of 28 minutes of literacy-focused time, a minimum of 16 minutes and 

a maximum of 52 minutes. All sessions included time for non-literacy activities, which included nonliteracy focused homework, math skill-building activities, and reward time. Sessions included an average 

of 16 minutes of non-literacy time, with a minimum of five minutes and a maximum of 32 minutes. On 

average, 64% of each session was spent on literacy related activities, which is an increase from 69% in 

2014-15. 

The variability in literacy versus non-literacy time sometimes was driven by the needs, abilities, and 

behavior of the student. For example, some students had a greater amount of non-literacy focused 

homework to complete. In other cases, the tutoring session structure was set by the tutor. For example, 

one tutor informed the observer that the student’s mother had requested that their session focus on 

getting homework, since the student was behind. Another example of this is a session in which the tutor 

alerted the observer to attention-deficit limitations for a student in which their sessions are generally 

shorter. Additionally, some tutors chose to spend a greater amount of time participating in free-time 

with the student. 

The amount of time allocated to different tutoring session components also varied between observed 

sessions. While the Bridge Project uses a flexible approach and the amount of time for each component 

should be set by the tutor-student pair based on goals, the tutoring model for grades Kindergarten 

through 3rd grade provides the following session guidelines to its tutors: 

• Check In 

• Homework or Skill Building (10 minutes) 

• GR8 Readers (25-30 minutes) 

• Game (preferably educational or art, as a reward, and there’s time left). 

Figure 7 depicts the average length in minutes of the five different components of tutoring sessions, by 

program site. On average, the amount of time spent on most activities varied greatly between sites, for 
 Prepared by OMNI Institute 

Page 19

all areas except for GR8 Readers book reading. Three of the four sites included homework time during 

the session, with the amount of time spent ranging from zero to 30 minutes. Additionally, the length of 

time spent on skill building activities was relatively low for most of the sites, whereas one session at site 

C was spent entirely on a skill building activity. At three of the four sites, reward time was almost 

comparable to time spent on GR8 Readers book reading. It is worth noting that reward time was never 

observed to be literacy focused and generally included games such as Jenga or Legos. 

Figure 7. Average Length of Different Session Components: Variation by Site 

9

15

30

4 4

45

11

15

18

22

12

9

6

13 12

8

9

0

5

10

15

20

25

30

35

40

45

Site A Site B Site C Site D

Minutes

Homework Time Skill-building Activity

GR8 Readers Book Reading GR8 Readers Packet Work

Reward Time

Literacy Skill Areas: The observer documented whether each tutoring session addressed each of five 

fundamental literacy skill areas. Figure 8 presents the percentage of sessions that included each of the 

five skill areas. Most tutoring sessions explicitly addressed fluency and phonemic awareness literacy 

skills. Vocabulary was covered in three-quarters of the observed sessions, phonics was addressed in 

67%, followed by comprehension which was covered in 58% of observed sessions. The percentages 

shown here are similar to the previous round of observations, except for vocabulary, in which there was 

an increase from 27% of sessions in 2014-15 to 75% in 2015-16. 
 Prepared by OMNI Institute 

Page 20

Figure 8. Percent of Tutoring Sessions that Explicitly Addressed Each Literacy Skill 

Tutoring Session Environment: Each of the four program sites has a designated room with tables 

and chairs that serves as a tutoring and study room. Each tutoring room also includes storage for 

tutoring folders/materials, reference books, games, and office supplies. While each site has a designated 

space, there were two sessions at one site where there were too many pairs for the room, and some 

were required to sit on the floor in the lobby, which proved to be very distracting for the pairs with 

other students passing by and the lack of hard surface for writing. 

While the observer primarily focused on the interactions between the tutor and the student, she also 

watched for factors within the tutoring environment that might interfere with the session, particularly 

environmental issues that served to distract students from session activities. During 11 of the 12 

observations, there were events that served as distractions from the tutoring session, ranging from one 

to four instances per observation. The majority of distracting events were excessive noise from other 

students and staff that were in close proximity, the presence of materials such as markers and rulers 

that were within reach, or reward time games that were picked out at the beginning of the session. 

However, in general, students rarely became distracted from tutoring sessions to such a degree that the 

distraction interfered with session activities. Further, when students did become distracted, tutors were 

generally able to quickly redirect students’ attention and keep the lesson on track. 

Student Learning and Behaviors: During those sessions where literacy materials were covered, all 

students demonstrated learning of literacy skills. The majority of observed students exhibited 

understanding of phonics, phonemic awareness, and fluency in tutoring sessions by effectively sounding 

out words and reading passages quickly and fluidly. Students also demonstrated learning of vocabulary 

and comprehension skills through correctly answering tutors’ questions, completing follow-up activities, 

and coming up with questions on the material themselves. Ten of the 12 observed students displayed no 

behavioral issues in the sessions. One student continually left the study area to walk around the room or 

leave the room to talk to other students and teachers. The tutor repeatedly attempted to refocus the 

student but the efforts did not prove to be effective. 
 Prepared by OMNI Institute 

Page 21

Instructional Skill of Tutors: The observer also collected data on the instructional skill of tutors. 

Overall, tutors provided students with level-appropriate materials, were engaged in sessions, had 

positive interactions with students, and generally practiced effective behavior management. Tutors 

showed some variability in the instructional pace they established for tutoring sessions, which was 

mostly driven by student need. For example, some sessions used additional time to complete 

homework, whereas others spent most of the time with the GR8 Readers book. There were also 

differences between tutors in the instructional strategies they implemented to assist, engage and 

challenge students. Many tutors allowed enough time for students to come to conclusions on their own, 

while some were quick to offer solutions. Additionally, for those tutors that encouraged the students to 

arrive at their own conclusions they would ask them to talk through their process, show them “proof” in 

the materials, or as them to think of another way to approach the problem. On numerous occasions, this 

strategy kept students engaged in the lesson. 

Tutoring Implementation Strengths and Challenges 

Programming strengths that were observed this year were that most sessions ran smoothly, students 

were engaged with the tutor and materials, and environmental distractions were observed infrequently. 

In general, this was an improvement from what was observed during the 2014-15 academic year in 

which student engagement and distractions were a challenge in keeping students engaged and the 

session running smoothly. Observation findings also highlighted an opportunity for the Bridge Project to 

continue improving the overall quality and consistency of its tutoring program. Recommendations based 

on those include helping tutors identify appropriate pace for non-GR8 Readers material and promoting 

the use of effective and innovative instructional strategies. It is worth noting that after the 2014-15 

round of tutoring observations, discussions with Bridge staff explained that tutoring sessions are often 

driven by student need, which can change the structure of each session, therefor consistency can be 

difficult to measure. Additionally, 12 sessions observed represent only a small proportion of the tutoring 

sessions offered by the program and may not be representative of all sessions, and we would expect 

some variation. 

Impact Study Results

Research Questions 

The impact study examines both confirmatory and exploratory research questions. The original proposal 

included three confirmatory research questions: 

4. Do more Bridge children show greater improvements in reading at or above grade level than 

comparison children at the end of each year and at the end of the study (binary DRA-2)? 
 Prepared by OMNI Institute 

Page 22

5. Do Bridge children show greater reading gains than comparison children, within each academic year 

and across all years of the study (truncated DRA-2)?

4

 

6. Do Bridge children show greater improvement in rates of school attendance than comparison 

children, within each academic year and across all years of the study (secondary outcome)? 

The proposal also included five exploratory research questions: 

1. Are more Bridge children reading at or above grade level by the end of third grade than 

comparison children? 

2. Does the amount or type of literacy programming predict reading gains? For example, does the 

number of years of participation in the program predict reading gains? 

3. What is the strength of the relationship between number of hours of participation and reading 

gains? Does this differ across program component? 

4. Which literacy program activities are most closely associated to reading gains; what are the 

critical program elements for success? 

5. Are results for grades K-3 maintained for Bridge children who reach fourth or fifth grade during 

the study period? Do these Bridge children show greater long-term gains in reading 

achievement than comparison children? 

Method 

Study Sample 

The study enrolled 389 youth, collected through four cohorts of participants. However, not all the youth 

have literacy/reading outcome data. The following analysis was based on students for whom there was 

at least one DRA-2 score available over the course of the study. The final sample for analysis includes 

371 study participants, 263 of whom were served by the Bridge Project, and 108 of whom were enrolled 

into the comparison group. Within these data, there were still missing data on some variables and not 

all students had complete data across all time points. 

Family and child demographic data, as well as data describing the children’s home environment and 

parental expectations, were used to better understand treatment and comparison group characteristics, 

assess differences between the two groups, and identify a closely matched sample that adjusts for preexisting differences between the two groups. The DRA-2 Word Analysis (WA) Task (obtained in the fall) 

and DRA-2 reading scores obtained before the start of the program for the cohorts provide the baseline 

 

4

 For question 1b, we revised the analysis strategy to explore grade level equivalencies rather than conducting a 

truncated grade level analysis. We learned that teachers administer the full DRA-2 independent level in the spring, 

therefore, this truncated approach was no longer needed and was replaced with a measure of grade level 

discrepancy. 
 Prepared by OMNI Institute 

Page 23

measure of kindergarten and 1st-3rd grade student reading skills, respectively. DRA-2 reading scores 

obtained at the end of the intervention years provide the follow-up measures of students’ literacy skills. 

Sample Characteristics 

Table 7 presents the descriptive statistics for the complete sample. The data suggest an economically 

disadvantaged sample, with low parent income and education, a mean family size of approximately five 

people, and a substantial portion (27%) using a language other than English as a primary language. 

Families identifying as African American comprise over one-third of the sample (39%), and families 

identifying as Hispanic or Latino comprise approximately half of the sample (49%). The parents report 

high educational goals and expectations for their children, and the means for frequency of eating dinner 

together, the number of books, and time spent reading with children indicate parental involvement with 

the children. School attendance rates are high across the baseline and follow-ups. 

Most children began the study in kindergarten (41%). With few exceptions, these children have only the 

DRA-2 WA baseline (40%), as the DRA-2 independent reading level is not assessed until the end of the 

kindergarten year. Over half (57%) of the sample demonstrated proficiency on the DRA-2 baseline (N = 

171), while less than half (44%) of the sample demonstrated proficiency when the DRA-2 and DRA-2 WA 

baseline were combined (N = 287); this indicates that fewer children are considered proficient at the 

beginning of kindergarten on the WA measure than on the DRA-2 end-of-prior-year assessments that 

are used at the higher grades. 

TABLE 7 - DESCRIPTIVE STATISTICS FOR BASELINE MEASURES: FULL SAMPLE 

Measure N Minimum Maximum Mean SD Data Type 

Family Members 362 2 13 5.14 2.07 Ratio

Children 357 1 9 3.42 1.81 Ratio

Income 359 0 4 0.51 0.78 Ordinal

Free Lunch Eligible 317 0 1 0.98 0.15 Binary

Education 366 0 5 1.92 1.23 Ordinal

English 335 0 1 0.73 0.45 Binary

Moves 349 0 10 0.97 1.41 Ratio

Age 368 5 10 6.36 1.26 Ratio

Gender 371 0 1 0.49 0.50 Binary

African American 334 0 1 0.39 0.49 Binary

Asian American 334 0 1 0.11 0.31 Binary

White 334 0 1 0.20 0.40 Binary

Native American 334 0 1 0.04 0.21 Binary

Other Race 332 0 1 0.19 0.39 Binary

Hispanic/Latino 346 0 1 0.49 0.50 Binary

Grade 371 0 3 1.16 1.16 Ratio

Attended Preschool 327 0 1 0.67 0.47 Binary

Dinner Together 359 0 3 2.45 0.81 Ordinal

Education Goals 358 0 5 4.53 1.12 Ordinal

Education Expect 342 1 5 4.61 0.87 Ordinal

Books 338 0 300 33.34 42.04 Ratio
 Prepared by OMNI Institute 

Page 24

Measure N Minimum Maximum Mean SD Data Type 

Read 328 0 3 1.92 0.96 Ordinal

Cohort 1 371 0 1 0.26 0.44 Binary

Cohort 2 371 0 1 0.27 0.45 Binary

Cohort 3 371 0 1 0.37 0.48 Binary

Cohort 4 371 0 1 0.09 0.29 Binary 

K at Baseline 371 0 1 0.41 0.49 Binary 

Grade 1 at Baseline 371 0 1 0.20 0.40 Binary 

Grade 2 at Baseline 371 0 1 0.20 0.40 Binary 

Grade 3 at Baseline 371 0 1 0.19 0.39 Binary 

DRA-2 Baseline 171 0 1 0.57 0.50 Binary 

DRA-2 Baseline + WA 287 0 1 0.44 0.50 Binary 

DRA-2 1-year 315 0 1 0.57 0.50 Binary 

DRA-2 2-Year 222 0 1 0.42 0.48 Binary 

DRA-2 3-year 58 0 1 0.48 0.50 Binary 

DRA-2 4-year 25 0 1 0.48 0.51 Binary 

Has WA Baseline Only 287 0 1 0.40 0.49 Binary 

School Attendance Baseline 200 0.42 1 0.91 0.10 Ratio 

School Attendance 1-year 354 0.49 1 0.93 0.08 Ratio 

School Attendance 2-year 283 0.14 1 0.94 0.09 Ratio 

School Attendance 3-year 136 0.70 1 0.95 0.05 Ratio 

School Attendance 4-year 42 0.59 1 0.94 0.07 Ratio 

Baseline Equivalence Analysis 

To more fully describe the sample, we examine differences between Bridge and comparison students at 

baseline. The analysis identifies the extent of pre-existing group differences and the need for 

adjustments in subsequent analyses. 

At baseline, Bridge students attended 51 different schools, and these schools served an average of five 

Bridge students (M=5.0, range 1-46). Students in the comparison group attended 18 different schools, 

and these schools served an average of approximately six comparison students (M=5.8, range 1-59). 

There was substantial overlap in schools between Bridge and comparison students. At the school level, 

11 of the 18 schools that comparison students attended at baseline also served Bridge students. At the 

student level, the majority (71%) of comparison students attended schools that Bridge students also 

attended. The substantial overlap in schools between Bridge and comparison students, despite being 

recruited from separate public housing communities, is likely related to the choice system employed by 

DPS. 

To test for differences between Bridge and comparison students, we compared group means for interval 

and ordinal variables and group proportions for binary variables. Table 8 lists the group means or 

proportions for each variable and the p-value for the t-test for differences in means or proportions. To 

adjust for multiple tests, we used the Holm-Bonferroni method to control the family-wise error rate. The 
 Prepared by OMNI Institute 

Page 25

standardized mean difference (Cohen’s d) indicates the effect size of the group differences, with the 

common rule of thumb that .2 equals a small difference, .5 a medium difference, and .8 a large 

difference. 

As expected with the design, the conditions are not equivalent at the start. Comparison of baseline 

means across conditions shows numerous statistically significant differences, most with small or 

medium effect sizes. Students in the Bridge condition seem “better off” than comparison students in 

some ways; they have made fewer moves (d = -.43), and their parents have higher educational goals and 

expectations (d = .49-.52). In other ways, Bridge students appear to face disadvantages relative to the 

comparison students; parents are less likely to speak English as their primary language (d = -.69), have 

fewer books in the home (d = -.50), and read to children less often (d = -.39). Additionally, there are 

some demographic differences; the Bridge students are younger (d = -.55), are in earlier grades (d = -

.38), and correspondingly are more likely to have the WA score as their baseline measure (d = .57); they 

are also more common in Cohorts 1 and 4 (ds = .42-.43), and less common in Cohort 3 (d = -.56) relative 

to the comparison students. Additionally, Bridge students are also more likely to be Asian American than 

comparison students (d = .32). Although the groups differ on numerous measures, the direction of the 

differences suggests little systematic bias. The mean of the d values across all measures in Table 8 is only 

-.02. 

TABLE 8 - DESCRIPTIVE STATISTICS AT BASELINE: FULL SAMPLE BY STUDY CONDITION 

Variable Condition N Means SD p d 

Family Members Comparison 99 5.01 2.24 0.453 0.08 

Bridge 263 5.19 2.01 

Children Comparison 96 3.48 1.95 0.749 -0.04 

Bridge 261 3.41 1.76 

Income Comparison 98 0.33 0.71 0.004 0.33 

Bridge 261 0.58 0.80 

Free Lunch Eligible Comparison 86 1.00 0.00 0.004 -0.25 

Bridge 231 0.97 0.17 

Education Comparison 106 2.05 1.06 0.008 -0.15 

Bridge 260 1.87 1.29 

English Comparison 95 0.92 0.28 0.000* -0.69 

Bridge 240 0.65 0.48 

Moves Comparison 95 1.46 1.97 0.002* -0.43 

Bridge 254 0.78 1.07 

Age Comparison 105 6.84 1.26 0.000* -0.55 

Bridge 263 6.16 1.20 

Gender Comparison 108 0.50 0.50 0.715 -0.04 

Bridge 263 0.48 0.50 

African-American Comparison 96 0.45 0.50 0.171 -0.16 

Bridge 238 0.37 0.48 

Asian-American Comparison 96 0.04 0.20 0.002* 0.32 

Bridge 238 0.13 0.34 

White Comparison 96 0.20 0.40 0.993 0.00 
 Prepared by OMNI Institute 

Page 26

Variable Condition N Means SD p d 

Bridge 238 0.20 0.40 

Native American Comparison 96 0.09 0.29 0.032 -0.26 

Bridge 238 0.03 0.16 

Other Race Comparison 94 0.16 0.37 0.380 0.10 

Bridge 238 0.20 0.40 

Hispanic Comparison 96 0.54 0.50 0.247 -0.14 

Bridge 250 0.47 0.50 

Grade Comparison 108 1.46 1.13 0.001* -0.38 

Bridge 263 1.03 1.14 

Preschool Comparison 75 0.57 0.50 0.063 0.25 

Bridge 252 0.69 0.46 

Dinner Comparison 100 2.40 0.86 0.435 0.08 

Bridge 259 2.47 0.79 

Education Goals Comparison 101 4.07 1.52 0.000* 0.52 

Bridge 257 4.71 0.85 

Education 

Expectations 

Comparison 88 4.25 1.20 0.001* 0.49 

Bridge 254 4.73 0.68 

Books Comparison 85 52.26 65.03 0.001* -0.50 

Bridge 253 26.99 28.16 

Read Comparison 106 2.19 0.91 0.001* -0.39 

Bridge 261 1.82 0.97 

Cohort 1 Comparison 108 0.14 0.35 0.000* 0.42 

Bridge 263 0.31 0.46 

Cohort 2 Comparison 108 0.28 0.45 0.878 -0.02 

Bridge 263 0.27 0.44 

Cohort 3 Comparison 108 0.56 0.50 0.000* -0.56 

Bridge 263 0.29 0.46 

Cohort 4 Comparison 108 0.02 0.14 0.000* 0.43 

Bridge 263 0.13 0.33 

DRA-2 Baseline Comparison 62 0.53 0.50 0.489 0.12 

Bridge 109 0.59 0.49 

DRA-2 Baseline + 

WA

Comparison 79 0.47 0.50 0.539 0.08 

Bridge 208 0.43 0.50 

Has WA Baseline 

Only 

Comparison 79 0.22 0.41 0.000* 0.57

Bridge 208 0.48 0.50 

School Attendance 

Baseline 

Comparison 72 0.90 0.11 0.235 0.19 

Bridge 128 0.92 0.10 

Note: Statistical significance (indicated by *) was determined using the Holm-Bonferroni Method to

account for multiple comparisons. Using this method, p-values of .004 and greater were not 

considered significant. 

Propensity Score Matching 

The differences between students in the two conditions require some form of adjustment. Controlling 

for baseline covariates in multiple regression models offers one approach. By including all students, 
 Prepared by OMNI Institute 

Page 27

however, the adjustments are strongly influenced by non-comparable extremes in each group. The 

alternative approach of propensity score matching identifies a subset of sample respondents in each 

condition who are most alike. The matching assumes all relevant differences between conditions are 

measured at baseline and limits the analysis by the loss of students and statistical power. Still, it has 

clear advantages in internal validity over use of all students.

We conducted propensity score matching using logistic regression for the outcome of Bridge (1) versus 

comparison (0) students with a nearest neighbor algorithm, no replacement, and one-to-one matches. 

The procedure uses the logistic regression predicted probabilities of belonging to the treatment group 

(i.e., propensity scores) for each subject. To improve the matching, we added two restrictions: cases 

were matched only if, first, they fell within the region of common support and, second, the caliper (or 

propensity score distance) for a match was less than or equal to .20 standard deviations. The propensity 

score matching was done with PSMATCHING3 in SPSS v. 21. 

All of the baseline measures listed in Table 8 were initially considered as predictors. These included 

measures of parental education and parental expectations for their children’s academic success, as 

these have been linked in previous research to children’s academic achievement (Child Trends Data 

Bank, 2012a; 2012b). These also included ethnic and racial identifiers, as research has shown ethnic and 

racial disparities in educational achievement (Hemphill & Vanneman, 2011; Vanneman, Hamilton, 

Baldwin Anderson, & Rahman, 2009). Additionally, data obtained from the Denver Housing Authority 

indicated that there may be differences between Bridge and comparison neighborhoods with respect to 

mobility, so a measure of mobility was also included. Finally, descriptive indicators of study involvement 

(i.e., cohort) were included to control for any unintended differences in recruitment from year to year. 

The first model, which considered all of these baseline measures, resulted in an n of 61 (just 16% of the 

sample) due to missing data across the many measures. Therefore, in order to increase sample size, 

measures needed to be removed as predictors. To determine which measures could be removed while 

limiting the loss of accuracy in propensity score matching, a number of steps were used. First, 

correlations greater than .80 were identified among all measures, and the measure with the higher level 

of missing data was removed. Such high correlations indicate that these measures would be capturing 

similar sources of variance, and the benefit of improving sample size would offset anything lost by 

excluding those measures. Second, the remaining measures with 10% or more missing data that also 

demonstrated significant mean differences between Bridge and comparison students were identified, 

and dummy variables that represented missingness (1=missing, 0=not missing) for those measures were 

included in the model as well. The resulting propensity score model yielded 120 matched cases (60 in 

each condition).5

 

5

Two alternative approaches were used to generate a matched sample. First, the same matching procedure 

described above was used, but mean replacement of missing values was used for those measures with 10% or 

more missing data and with a mean difference between Bridge and comparison students. The resulting propensity 
 Prepared by OMNI Institute 

Page 28

Table 9 presents tests for baseline equivalence for the subsample of 120 students. Given missing data, 

the tests often rely on fewer cases. Based on the results, the matched sample appears relatively well 

balanced; the only statistically significant differences are in parental educational expectations, with 

Bridge students’ parents with higher educational expectations (d = .64), and in the proportion of the 

sample drawn from Cohort 3, with fewer Bridge than comparison students in the matched sample from 

Cohort 3 (d = -.82). Several other measures show d values in the medium range but are not statistically 

different; the Bridge group speaks English less often (d = -.49), is younger (d = -.42), and has higher 

educational goals (d = .47). 

TABLE 9 - DESCRIPTIVE STATISTICS AT BASELINE: MATCHED SAMPLE 

Variable Condition N Mean SD p d 

Family Members Comparison 60 4.88 1.83 0.231 0.22 

Bridge 60 5.30 1.96 

Children Comparison 58 3.52 1.73 0.797 0.05 

Bridge 60 3.60 1.75 

Income Comparison 57 0.40 0.86 0.146 0.27 

Bridge 60 0.65 0.95 

Free Lunch Eligible Comparison 46 0.00 0.00 0.159 0.27 

Bridge 57 0.04 0.19 

Education Comparison 60 2.08 0.94 0.206 -0.23 

Bridge 60 1.83 1.20 

English Comparison 51 0.90 0.30 0.013 -0.49 

Bridge 56 0.71 0.46 

Moves Comparison 60 1.10 1.46 0.486 -0.13 

Bridge 60 0.93 1.13 

Age Comparison 60 6.82 1.14 0.024 -0.42 

Bridge 60 6.33 1.17 

Gender Comparison 60 0.55 0.50 0.204 -0.23 

Bridge 60 0.43 0.50 

African-American Comparison 53 0.30 0.46 0.726 0.07 

Bridge 57 0.33 0.48 

Asian-American Comparison 53 0.02 0.14 0.349 0.18 

Bridge 57 0.05 0.23 

White Comparison 53 0.28 0.45 0.382 -0.17 

 

score model yielded 110 matched cases (55 in each condition) with two significant mean differences between 

Bridge and comparison students (using the Holm-Bonferroni correction) on baseline measures for this subsample; 

given that the two samples appeared to be equally well-matched, the larger matched sample was retained. 

Second, the same matching procedure described above was used, but the model dropped those measures with 

10% or more missing data and with a mean difference between Bridge and comparison students. The resulting 

propensity score model yielded 128 matched cases (64 in each condition) with five significant mean differences 

between Bridge and comparison students (using the Holm-Bonferroni correction) on baseline measures for this 

subsample; therefore, the smaller but better-matched sample was retained. 

 
 Prepared by OMNI Institute 

Page 29

Variable Condition N Mean SD p d 

Bridge 57 0.21 0.41 

Native American Comparison 53 0.08 0.27 0.158 -0.28 

Bridge 57 0.02 0.13 

Other Race Comparison 53 0.17 0.38 0.591 0.10 

Bridge 57 0.21 0.41 

Hispanic Comparison 56 0.66 0.48 0.122 -0.29 

Bridge 58 0.52 0.50 

Grade Comparison 60 1.48 1.11 0.299 -0.19 

Bridge 60 1.27 1.16 

Preschool Comparison 44 0.61 0.49 0.147 0.29 

Bridge 60 0.75 0.44 

Dinner Comparison 60 2.48 0.72 0.494 -0.13 

Bridge 60 2.38 0.87 

Education Goals Comparison 59 4.19 1.35 0.013 0.47 

Bridge 58 4.71 0.82 

Education Expect Comparison 52 4.06 1.26 0.001* 0.64 

Bridge 58 4.72 0.74 

Books Comparison 60 37.83 37.68 0.966 -0.01 

Bridge 60 37.55 34.10 

Read Comparison 60 2.13 0.95 0.779 -0.05 

Bridge 60 2.08 1.00 

Cohort 1 Comparison 60 0.23 0.43 0.076 0.33 

Bridge 60 0.38 0.49 

Cohort 2 Comparison 60 0.22 0.42 0.107 0.30 

Bridge 60 0.35 0.48 

Cohort 3 Comparison 60 0.55 0.50 0.000* -0.82 

Bridge 60 0.18 0.39 

Cohort 4 Comparison 60 0.00 0.00 0.024 0.42 

Bridge 60 0.08 0.28 

DRA-2 Baseline Comparison 36 0.44 0.50 0.130 0.38 

Bridge 30 0.63 0.49 

DRA-2 Baseline + WA Comparison 46 0.39 0.49 0.654 0.09 

Bridge 48 0.44 0.50 

Has WA Baseline Only Comparison 46 0.22 0.42 0.096 0.35 

Bridge 48 0.38 0.49 

School Attendance Baseline Comparison 36 0.89 0.11 0.634 0.12 

Bridge 27 0.91 0.11 

Note: Statistical significance (indicated by *) was determined using the Holm-Bonferroni Method to 

account for multiple comparisons. Using this method, p-values of .004 and greater were not considered 

significant. 

Differential Attrition and Missing Data Analysis 

The sample contains substantial missing data due to attrition, non-response to survey items, and 

inability to match students with school records. Other gaps in data result from the timing of the surveys. 
 Prepared by OMNI Institute 

Page 30

Given their later entrance into the study, Cohort 4 students have data for the 1-year follow-up, but are 

missing data for the 2-year, 3-year, and 4-year follow-ups. Cohort 3 students have data for the 1-year 

and 2-year follow-ups but are missing the 3-year and 4-year follow-ups. Cohort 2 students have data for 

the 1-year, 2-year, and 3-year follow-ups but are missing the 4-year follow-up. Cohort 1 students have 

data for all the follow-ups, but are missing baseline school attendance data, as these data were not 

available before the start of the study. 

Table 10 presents figures on the completeness of data for the DRA-2 reading and school attendance 

outcome measures. The values in Table 10 represent the proportion of missing data, and only include 

participants that are eligible for each follow-up (e.g., only Cohort 1 participants are included in the 

calculations of attrition for the 4-year follow-up). Of the 371 participants, the DRA-2 baseline scores 

were missing for 54% but the combined DRA-2 + WA scores were missing for 23% of the full sample. The 

follow-up DRA-2 scores were missing relatively little data at 1-year (15%), but the amount of missing 

data naturally increases at the later follow-ups, to 34% at the 2-year follow-up, 71% at the 3-year followup, and 74% at the 4-year follow-up for the full sample. Values for the matched sample show slightly 

smaller proportions of missing data at each of the follow-ups. 

The next columns in Table 10 examine missing data for the 262 students with the DRA-2 + WA baseline. 

The figures are important for models including the baseline measure as a predictor of the follow-up 

measures. The proportions missing are generally lower for this subsample, with the exception of the 2-

year and 3-year follow-up attendance data, where missing data rates are slightly higher. 

TABLE 10 - PROPORTIONS OF MISSING DATA 

With or Without Pretest Students with DRA-2 + WA Pretest 

Full Sample Matched Sample Full Sample Matched Sample 

DRA-2 Baseline 0.54 0.45 0.40 0.30 

Baseline + WA 0.23 0.22 0.00 0.00 

DRA-2 1-year 0.15 0.13 0.08 0.07 

DRA-2 2-Year 0.34 0.30 0.31 0.32 

DRA-2 3-year 0.71 0.68 0.63 0.61 

DRA-2 4-year 0.74 0.76 0.74 0.78 

School Attendance Baseline 0.27 0.24 0.00 0.00 

School Attendance 1-year 0.05 0.03 0.03 0.00 

School Attendance 2-Year 0.16 0.10 0.17 0.15 

School Attendance 3-year 0.31 0.25 0.41 0.38 

School Attendance 4-year 0.57 0.41 - - 

Differential attrition was tested in two steps. First, to examine baseline differences between those with 

complete and missing data at each follow-up, we correlated the baseline measures listed in Table 9 with 

measures equal to 1 for those without DRA-2 outcome data at each follow-up and equal to 0 for those 

with DRA-2 outcome data. The four measures of DRA-2 missingness for each of the follow-ups and the 
 Prepared by OMNI Institute 

Page 31

30 baseline measures produce 111 valid correlations6, 21 of which were significant at the .05 level; using 

the Holm-Bonferroni method to correct for multiple comparisons, only four were significant. These

findings point to minimal differences in attrition. 

Second, the more complete test of differential attrition comes from tests for the interaction, or product 

term, of condition by the baseline measure in predicting missingness. For attrition to bias tests for 

program impact, attrition related to baseline characteristics must also differ by condition. We therefore 

estimated 21 such logistic regression models, using the less conservative benchmark of p < .05 to probe 

for possible differences in attrition. Results suggest that there are two baseline measures which differ in 

missingness between conditions: Bridge students in Cohort 3 were more likely to drop out at the 2-year 

follow-up than comparison students in Cohort 3 (p = .044); and Bridge students with low school 

attendance were more likely to be missing data at the 2-year follow-up than comparison students with 

low school attendance (p = .028). However, using the Holm-Bonferroni adjustment, neither of these 

findings are considered statistically significant. 

Among the matched sample, 14 of 111 correlations reached statistical significance, again using the less 

conservative benchmark of p < .05; using the Holm-Bonferroni method to correct for multiple 

comparisons, only three were significant. Further, none of the interaction terms in the subsequent 

logistic regression models to probe for possible differences in attrition was significant using the p < .05 

benchmark. Overall, given the numerous baseline characteristics, the tests for differential attrition do 

not indicate a systematic problem. The influence of being a member of Cohort 3 and school attendance 

are considered as control variables in analyses of the full sample that follow, but attrition is viewed 

more as a problem of sample size and generalizability than a threat to the internal validity of the 

program evaluation. 

Power Analysis 

The maximum sample size for the full sample at the 1-year follow-up is 332 (102 comparison, 230 

treatment), but only 250 have both baseline and posttest data for DRA-2 proficiency (73 comparison, 

177 treatment), and only 195 have both baseline and 1-year follow-up data for school attendance (70 

for comparison, 125 for treatment). The matched sample has a maximum sample size of 120 (60 each 

for the comparison and treatment). 

Power analyses with power of .80 and an alpha level of .05 indicate minimal detectable effect sizes 

(Cohen’s d) ranging from .33-.42 for these sample sizes with a standard regression model. This indicates 

that the data have the potential to identify small-to-medium effect sizes. To translate the effect sizes 

into odds ratios appropriate for logistic regression, we use the standard formula of OR = exponent 

(1.81*d). The small-to-medium standardized effect sizes of .33-.42 correspond to odds ratios of 1.82-

 

6

 Of the 120 possible correlations, nine were not calculated because at least one of the variables was constant 

(e.g., all of Cohort 4 is missing 2-year, 3-year, and 4-year follow-up data). 
 Prepared by OMNI Institute 

Page 32

2.13. For the matched sample, power analyses indicate minimal detectable effect size of .52 – a medium 

effect size. The corresponding odds ratio is 2.56. 

Descriptive Outcomes 

Preliminary descriptive analyses were conducted to examine mean levels of DRA-2 proficiency and 

school attendance at baseline and follow-ups. The top panel of Table 11 lists descriptive statistics for the 

DRA-2 reading proficiency figures at baseline and the 1-year and 2-year follow-ups for the full sample. 

There were too few cases for analysis of DRA-2 proficiency at the 3-year follow-up (Comparison n = 6; 

Bridge n = 46) and the 4-year follow-up (Comparison n = 4; Bridge n = 18). The bottom panel of Table 11 

lists descriptive statistics for the school attendance rates at baseline and the 1-year and 2-year followups for the full sample. Once again, there were too few cases for analyses of school attendance at the 3-

year follow-up (Comparison n = 20; Bridge n = 17) and there were no data for the 4-year follow-up.

The outcome measures for the DRA-2 indicate whether the student is rated proficient (equal to 1) or not 

(equal to 0). The baseline scores combine the Word Analysis (WA) proficiency score for those in 

kindergarten with the DRA-2 proficiency score for those in grades 1-3 at the program start. The use of 

the same scale by the DRA-2 and WA measures – proficient or not – allows for the combination across 

these measures. The table examines students with both baseline and follow-up scores. 

The 1-year follow-up means and associated effect sizes in Table 11 for DRA-2 reading proficiency suggest 

that Bridge students fare better over time than comparison students. At baseline, Bridge students score 

lower than comparison students (41% proficient versus 51% proficient, respectively; d = -.20), while at 

follow-up, Bridge students score better than comparison students (61% proficient versus 52% proficient, 

respectively; d = .18). A small amount of growth occurs among Bridge students during the first year 

(change = .20), while little change occurs among comparison students (change = .01), and this 

difference is statistically significant (p < .05). These differences appear to persist at the 2-year follow-up. 

After two years, Bridge students continue to score better than comparison students on average (49% 

proficient versus 33% proficient, respectively; d = .33). Additionally, at two years the Bridge students 

again show greater positive change (change = .09) than comparison students (change = -.18), and this 

difference in change scores is statistically significant (p < .05). 

TABLE 11 - BASELINE AND FOLLOW-UP OUTCOMES: FULL SAMPLE 

DRA-2 Proficiency 

Baseline 

(DRA-2 + WA) 

Follow-up Change Sig 

Comp Bridge Comp Bridge Comp Bridge 

1-year follow-up N 73 189 73 189

(Cohorts 1-4) Mean 0.51 0.41 0.52 0.61 0.01 0.20 **

SD 0.50 0.49 0.50 0.49

d -0.20 0.18

2-Year follow-up N 55 123 55 123

(Cohorts 1-3) Mean 0.51 0.40 0.33 0.49 -0.18 0.09 **
 Prepared by OMNI Institute 

Page 33

SD 0.50 0.49 0.47 0.50

d -0.22 0.33

School Attendance 

Baseline Follow-up Change Sig 

Comp Bridge Comp Bridge Comp Bridge 

1-year follow-up N 70 125 70 125

(Cohorts 2-4) Mean 0.90 0.92 0.91 0.94 0.01 0.02

SD 0.11 0.09 0.10 0.07

d 0.20 0.35

2-Year follow-up N 66 75 66 75

(Cohorts 2-4) Mean 0.91 0.94 0.92 0.95 0.01 0.01

 SD 0.10 0.07 0.08 0.06 

d 0.35 0.43 

* p < .10, ** p < .05, *** p < .01 

The results for school attendance with the full sample do not reveal any significant differences between 

Bridge and comparison students, as there is no significant difference in the change in school attendance 

from baseline to first-year or second-year follow-up among either group. For both Bridge and 

comparison students, school attendance appears to stay relatively stable on average. 

Table 12 lists descriptive statistics for the DRA-2 reading proficiency figures at baseline and the 1-year 

and 2-year follow-ups for the matched sample. Once again, there were too few cases for analysis of 

DRA-2 proficiency at the 3-year follow-up (Comparison n = 5; Bridge n = 9) and there were no 

comparison students in the 4-year follow-up. Likewise, there were too few cases for analyses of school 

attendance at the 3-year follow-up (Comparison n = 11; Bridge n = 4) and there were no data for the 4-

year follow-up.

Results with the matched sample indicate greater improvement for Bridge than comparison students in 

DRA-2 proficiency, particularly by the 2-year follow-up. At baseline, comparison students score slightly 

lower than Bridge students (42% proficient versus 45% proficient, respectively; d = .07). At the 1-year 

follow-up, the Bridge students score better than the comparison students (59% proficient versus 49% 

proficient, respectively; d = .20), although this difference is not statistically significant. However, by the 

2-year follow-up, the Bridge students continue to score better than the comparison students (48% 

proficient versus 25% proficient, respectively; d = .49), and the difference in change scores is statistically 

significant (p < .05). 

TABLE 12 - BASELINE AND FOLLOW-UP OUTCOMES: MATCHED SAMPLE 

DRA-2 Proficiency 

Baseline 

(DRA-2 +WA) 

Follow-up Change Sig 

Comp Bridge Comp Bridge Comp Bridge 

1-year follow-up N 43 44 43 44

(Cohorts 1-4) Mean 0.42 0.45 0.49 0.59 0.07 0.14

 SD 0.50 0.50 0.51 0.50 
 Prepared by OMNI Institute 

Page 34

d 0.07 0.20 

2-Year follow-up N 32 29 32 29

(Cohorts 1-3) Mean 0.44 0.38 0.25 0.48 -0.13 0.10 **

 SD 0.50 0.49 0.44 0.51

d -0.12 0.49

School Attendance 

Baseline Follow-up Change Sig 

Comp Bridge Comp Bridge Comp Bridge 

1-year follow-up N 36 27 36 27

(Cohorts 2-4) Mean 0.89 0.91 0.91 0.94 0.02 0.03 

SD 0.11 0.11 0.07 0.08

d 0.12 0.32 

2-Year follow-up N 34 17 34 17

(Cohorts 2-4) Mean 0.90 0.92 0.93 0.93 0.03 0.01 

SD 0.11 0.05 0.05 0.08

D 0.30 0.01

* p < .10, ** p < .05, *** p < .01 

The results for school attendance with the matched sample do not indicate any significant differences 

between Bridge and comparison students, as there is no significant difference in the change in school 

attendance from baseline to first-year or second-year follow-up among either group. For both Bridge 

and comparison Bridge students, school attendance appears to stay relatively stable on average. 

Multiple Regression Analyses 

To examine the confirmatory research questions of interest, multiple regression analyses were 

conducted. Specifically, we estimated logistic regression models for the binary DRA-2 proficiency 

measure, and linear regression models for school attendance rate. To help interpret the size of the 

effects, the tables present odds ratios for the logistic regressions and standardized coefficients for the 

linear regressions. We estimated separate models for each of the follow-ups, where there was a 

sufficient sample size to do so. Analyses were conducted in Mplus using full information maximum 

likelihood (FIML) to account for missing data. FIML uses a latent variable approach to handle missing 

data in endogenous variables (i.e., independent variables) within the analysis model, which results in all 

possible cases (i.e., all those with an outcome variable) being used for estimation. As a result, FIML is a 

more powerful estimation model than list-wise deletion approaches because it results in a larger sample 

size and is less biased by patterns of attrition. In Mplus, logistic regressions were conducted using 

maximum likelihood estimation with robust standard errors, which adjusts estimates to account for nonnormally distributed data. Prior to the analyses we considered two statistical concerns that accompany 

the quasi-experimental design of the Bridge Project: (1) clustering of the sample, and (2) nonrandomized assignment across conditions. 

With respect to the clustering of the sample, the students are clustered within housing complexes, 

which are the primary basis for assignment to the treatment group. As described earlier, assignment to 

the treatment is dependent on the housing location of students, such that assignment is based on 
Prepared by OMNI Institute

Page 35

groups rather than individuals. Therefore, we considered whether it was necessary to account for 

variation in the outcome across units of assignment via mixed-effects models by examining the 

distribution of variance in the outcomes. ANOVA tests show that the housing complex in which the 

parents and child live accounts for less than 5% of variance in either DRA-2 proficiency or school 

attendance rates, indicating that a mixed-effect model approach is not necessary. Clustering does not 

appear to bias estimation of the standard errors.

Second, as demonstrated in preceding sections on baseline equivalence, the non-randomized 

assignment means the treatment and comparison groups are not equivalent before the start of the 

treatment. The simplest adjustment is to control for baseline characteristics that differ across groups. Of 

all the measures that differ across condition or are affected by attrition for the full sample, five are 

related independently to the DRA-2 outcome and therefore may confound the effects of the treatment: 

age (p = .004), grade (p = .005), whether the DRA-2 WA is used (p < .001), and speaking English at home

(p = .049) at baseline. Of note, three of these measures are highly interrelated; age is highly related to 

grade, and students in Kindergarten at baseline have the DRA-2 WA score as their baseline measure. For 

school attendance, the only significant baseline measure is number of moves (p < .001). The models for 

the full sample therefore include all these covariates and should help minimize potential confounding. 

In addition, as described earlier, the more stringent and complex form of control involves propensity 

score matching. We replicate the impact models for the subsample of 120 matched cases, sacrificing 

sample size and power for equivalence across conditions. The baseline difference between the paired 

sample in number of moves prior to baseline is included as a covariate to account for possible 

confounding effects. For both the full and matched samples, several of these models are limited by a 

small number of cases for longer follow-ups. In those cases, analyses are not conducted. 

Effects of Bridge Program on Outcomes

Table 13 presents results from multiple regression models at the 1-year, 2-year, and 3-year follow-ups

for the full sample. There were too few cases for reliable analysis of DRA-2 proficiency (N = 24) and 

school attendance (N = 42) at the 4-year follow-up.

Results indicate that at the 1-year follow-up, Bridge students were not statistically more likely than 

comparison students to be proficient on the DRA-2. At the 2-year follow-up, Bridge students were more 

likely to be proficient on the DRA-2 than comparison students (OR = 2.32, p < .05). At the 3-year followup, Bridge students were marginally more likely to be proficient on the DRA-2 than comparison students 

(OR = 5.95, p < .10). As expected, baseline proficiency levels were positively related to DRA-2 proficiency 

at all follow-ups. With respect to the covariates, speaking English at home and age were negatively 

related to DRA-2 proficiency at the 1-year follow-up, and grade and having a DRA-2 WA as a baseline 

score were positively related to DRA-2 proficiency at the 1-year follow-up. At the 2-year follow-up, only 

speaking English at home continued to be negatively related to DRA-2 proficiency. At the 3-year followup, no other covariates were related to DRA-2 proficiency. The negative relation between speaking 

English at home and DRA-2 proficiency was unexpected, but replicated in post-hoc analyses, where 
 Prepared by OMNI Institute 

Page 36

students whose parents reported speaking English as their primary language had lower average levels of 

proficiency at the 1-year and 2-year follow-ups (ps < .05) 

TABLE 13 - EFFECTS OF BRIDGE PROGRAM ON OUTCOMES: FULL SAMPLE 

DRA-2 Proficiency Logistic Regression 

1-year follow-up 2-year follow-up 3-year follow-up

Predictors B OR Sig B OR Sig B OR Sig 

DRA-2 + WA Baseline 0.60 21.08 *** 0.49 8.71 *** 0.40 6.95 ***

Bridge -0.04 0.81 0.17 2.32 ** 0.33 5.95 * 

English -0.12 0.51 * -0.17 0.44 ** -0.09 0.62 

Age -0.38 0.46 ** 0.07 1.13 -0.33 0.53 

Grade 0.40 2.38 ** -0.21 0.68 0.06 1.12 

Baseline+WA 0.56 18.50 *** 0.04 1.20 0.13 1.92 

Constant -0.96 0.79 -0.47 

R

2 0.49 0.32 0.45 

N 315 222 58

School Attendance Linear Regression 

1-year follow-up 2-year follow-up 3-year follow-up

Predictors b beta Sig b beta Sig b beta Sig 

School Attendance Baseline 0.51 0.62 *** 0.59 0.71 *** 0.22 0.43 ***

Bridge 0.00 0.04 0.00 0.12 0.01 0.05 

Moves -0.02 -0.29 *** -0.01 -0.34 *** -0.00 -0.09 

Constant 0.48 0.40 0.74 

R

2 0.56 0.57 0.22 

N 354 283 136 

* p < .10, ** p < .05, *** p < .01 

Results also indicate that Bridge and comparison students did not differ in school attendance levels at 

any of the follow-ups. As expected, school attendance at baseline was positively related to school 

attendance at all follow-ups; additionally, the number of moves was negatively related to school 

attendance at the 1-year and 2-year follow-ups, but not the 3-year follow-up. 

Table 14 presents results from multiple regression models at the 1-year and 2-year follow-ups for the 

matched sample. There were too few cases for reliable analysis of DRA-2 proficiency or school 

attendance at the 3-year (Ns = 23 and 53, respectively) or 4-year (Ns = 9 and 22, respectively) follow-ups 

for the matched sample. Results indicate that there were no significant differences between Bridge and 

comparison students for DRA-2 proficiency at the 1-year follow-up, but Bridge students were more likely 

to be proficient than comparison students among the matched sample at the 2-year follow-up (OR=4.57, 

p < .05). 

TABLE 14 - EFFECTS OF BRIDGE PROGRAM ON OUTCOMES: MATCHED SAMPLE 

DRA-2 Proficiency Logistic Regression 

1-year follow-up 2-year follow-up

Predictors b OR Sig b OR Sig 
 Prepared by OMNI Institute 

Page 37

DRA-2 + WA Baseline 0.46 7.01 *** 0.45 8.67 ***

Bridge 0.15 1.88 0.32 4.57 **

Education Expectation -0.01 0.97 0.27 1.13 * 

Constant 0.58 2.13 

R

2

.25 .42

N 105 81

School Attendance Linear Regression 

1-year follow-up 2-year follow-up

Predictors b beta Sig b beta Sig 

Attendance Baseline 0.39 0.54 *** 0.62 0.82 ***

Bridge 0.01 0.05 0.00 0.01 

Education Expectation -0.01 -0.10 0.00 0.01 

Cohort 3 -0.00 -0.02 0.02 0.12 

Constant 0.60 0.35 3.67 

R

2 0.33 0.67 

N 117 104 

* p < .10, ** p < .05, *** p < .01 

With respect to covariates, parental education expectations were positively related to DRA-2 proficiency 

at the 2-year follow-up. There were no significant differences in school attendance between Bridge and 

comparison students at either the 1-year or 2-year follow-ups. Baseline school attendance was positively 

related to school attendance at both the 1-year and 2-year follow-ups, and none of the other covariates 

were significant. 

Linear Growth, Mixed-Effects Models 

The separate analysis of each follow-up ignores trajectories over time for students with multiple data 

points and fails to account for correlated errors across equations (i.e., does not account for the fact that, 

for the most part, the same students are contributing to the estimates in each model). A linear growth, 

mixed-effects (or multi-level) model incorporates long-term change by pooling data from the baseline 

and the four follow-ups. This has the additional advantage of using data from the third and fourth 

follow-ups, for which there is too small a sample size at each follow-up to model independently. The 

mixed-effects model treats time at level 1 and persons at level 2, with time nested within persons. By 

assuming a linear trajectory in outcomes over time, the model has the advantages of increasing the 

sample size and allowing use of students with partially complete data. The mixed-effects estimation 

adjusts for potential problems of clustering and heteroscedasticity that otherwise would bias estimates 

of the standard errors. 

Table 15 lists the model coefficients for the mixed-effects models. Note that the models add a measure 

of wave (0 through 4, corresponding to the baseline and each follow-up) and a wave-by-treatment 

interaction term to capture the growth in the outcomes. To adjust for the baseline differences across 

conditions, the models also include a baseline outcome control and the covariates used in the previous 

multiple regression models, which are constant across the waves. 
 Prepared by OMNI Institute 

Page 38

The mixed-effects models provide evidence of Bridge effectiveness for DRA-2 proficiency. The Bridge-bywave growth term is statistically significant (p < .01) for DRA-2 proficiency, indicating that change in 

proficiency over time differs between Bridge and comparison students. Probing this interaction by 

examining simple slopes indicates that the likelihood of being proficient increases significantly among 

Bridge students over time (b = .06, SE = .02, p = .003), while the likelihood of being proficient decreases 

significantly among comparison students (b = -.09, SE = .03, p = .001). 

Results for the matched sample indicate a similar trend; the Bridge-by-wave growth term is statistically 

significant (p < .05), indicating that change in proficiency over time differs between Bridge and 

comparison students in the matched sample as well. Probing of these simple slopes indicate that the 

likelihood of being proficient increases marginally among Bridge matched students over time (b = .07, SE 

= .04, p = .08), while the likelihood of being proficient remains stable for comparison matched students 

over time (b = -.06, SE = .04, p = .17). 

TABLE 15 - EFFECTS OF BRIDGE PROGRAM ON OUTCOMES: MIXED-EFFECTS MODELS 

Full Sample 

DRA-2 Proficiency School Attendance 

Intercept 0.36 0.29 

Bridge -0.06*** -0.00 

Wave -0.09*** 0.00 

Bridge*Wave 0.15*** 0.01 

DRA-2 0.78*** - 

English -0.04* - 

Age -0.04 - 

Grade 0.02 - 

Baseline+WA 0.12*** - 

Attendance Base - 0.70**

Move - -0.02* 

N 335 362 

Matched Sample 

DRA-2 Proficiency School Attendance 

Intercept 0.07 0.40 

Bridge 0.01 0.00 

Wave -0.05 0.01**

Bridge*Wave 0.12** 0.00 

DRA-2 0.77*** - 

Education Expectation 0.01 - 

Cohort 3 0.37 - 

Attendance Base - 0.57***

Move - -0.01 
 Prepared by OMNI Institute 

Page 39

N 117 120 

* p < .10, ** p < .05, *** p < .01

As shown in Figure 9, the probability that Bridge and comparison students are proficient on the DRA-2 at 

baseline is equal; however, by the end of the third year of programming, a Bridge student has a 71% 

chance of being proficient, while a matched comparison student has a 62% chance of being proficient. 

FIGURE 9. PROBABILITY OF PROFICIENCY OVER TIME AMONG PROPENSITY-SCORE MATCHED 

BRIDGE AND COMPARISON STUDENTS 

Note: The difference in slopes between the matched Bridge and comparison groups was statistically significant

For school attendance, the Bridge-by-wave growth term is not statistically significant for either the full 

or matched sample. For the matched sample there is a main effect of wave, indicating that school 

attendance rates increased on average over time, but there does not appear to be a difference between 

Bridge and comparison students in this growth. 

Deidentified Data 

Given that recruitment for the study did not reach initial benchmarks, resulting in a somewhat smaller 

sample size than originally anticipated, OMNI explored an additional research design using de-identified 

student data. For this process, OMNI identified the four most common home schools for Bridge 

students; among these four schools, 98 Bridge students were in school attendance (School 1 n = 33; 

School 2 n = 20; School 3 n = 27; School 4 n = 18). OMNI requested data from these schools for nonBridge students in grades K-3. This request resulted in data for a non-Bridge sample of 597 students 

(School 1 n = 155; School 2 n = 154; School 3 n = 85; School 4 n = 203). For these students, DRA-2 and 

school attendance data from the 2014-15 school year served as the baseline, and data from the 2015-16 
 Prepared by OMNI Institute 

Page 40

school year served as the 1-year follow-up. Due to restrictions from the school district, no demographic 

information other than grade could be collected for these students. 

Analysis of the deidentified sample once again raised concerns about the clustering of students, since 

the sample came intentionally from four specific schools. Therefore, we considered whether it was 

necessary to account for systematic variation in the outcome across schools via mixed models. However, 

ANOVA tests show that school accounted for less than 2% of variance in either DRA-2 proficiency or 

school attendance rates, indicating that a mixed model approach was not necessary. Clustering by 

schools likely does not bias estimation of the standard errors. 

The first set of analyses compared change in DRA-2 proficiency and school attendance from the 2014-15 

school year to the 2015-16 school year between Bridge and comparison students. Although previous 

tests indicated that schools do not account for a significant level of variance in either outcome, thus 

eliminating the need for mixed models, it is also important to consider whether there are mean 

differences between schools at baseline. Mean levels of DRA-2 proficiency and school attendance rates 

at baseline for each school are presented in Table 16. There were no significant differences among 

schools in baseline DRA-2 proficiency levels. For school attendance, School 2 had significantly lower 

rates than School 1 (p < .001) and School 4 (p = .001). Therefore, school was included as a covariate in 

subsequent school attendance analyses to control for mean differences using a series of three dummycoded variables. 

TABLE 16 - DESCRIPTIVE STATISTICS BY SCHOOL AT BASELINE: DEIDENTIFIED SAMPLE 

Variable School N Mean SD 

DRA-2 Baseline School 1 128 0.64 0.48 

School 2 116 0.51 0.50 

School 3 62 0.56 0.50 

School 4 146 0.58 0.50 

School Attendance Baseline School 1 137 0.94 0.07 

School 2 123 0.89 0.09 

School 3 83 0.91 0.12 

School 4 163 0.93 0.06 

Next, mean differences between Bridge and comparison students at baseline were examined to 

determine whether the samples were systematically different from the outset. As shown in Table 17, 

there were no significant differences between Bridge and comparison students in either baseline DRA-2 

baseline proficiency levels or attendance rates. 

TABLE 17 - DESCRIPTIVE STATISTICS BY CONDITION AT BASELINE: DEIDENTIFIED SAMPLE 

Variable Condition N Mean SD p Sig d 

DRA-2 Baseline Comparison 371 0.57 0.50 0.760 0.04 

Bridge 81 0.59 0.49 

School Attendance Baseline Comparison 415 0.92 0.08 0.512 0.11 

Bridge 91 0.93 0.10
 Prepared by OMNI Institute 

Page 41

* p < .10, ** p < .05, *** p < .01 

The baseline equivalence across Bridge and comparison students, as well as across schools, offers some 

assurance about the ability to meaningfully test for group differences in outcomes. For this, multiple 

regression models were conducted. The limited number of measures that were available for the 

comparison students in the deidentified sample limited the covariates that could be considered in these 

models. For the DRA-2 proficiency logistic regression model, DRA-2 baseline and school attendance from 

the previous year were included as covariates. For the school attendance linear regression model, 

baseline attendance and DRA-2 proficiency from the concurrent year were included as covariates. Table 

18 presents results from these multiple regression models. 

TABLE 18 - EFFECTS OF BRIDGE ON OUTCOMES: DEIDENTIFIED SAMPLE 

DRA-2 Proficiency Logistic Regression 

1-year follow-up

Predictors B OR Sig 

DRA-2 Baseline 0.57 13.36 ***

Bridge -0.06 0.70

Baseline School Attendance 0.09 9.11

Constant 1.29 

R

2

.34 

N 638 

School Attendance Linear Regression 

1-year follow-up

Predictors b beta Sig 

DRA-2 0.01 0.06

Bridge 0.03 0.11 ** 

Baseline School Attendance 0.33 0.33 ***

School 1 0.00 0.02 

School 2 -0.01 -0.06 

School 3 -0.02 -0.10 **

Constant 0.62 

R

2 0.16

N 644 

* p < .10, ** p < .05, *** p < .01 

Results indicate that there was no difference between Bridge and comparison students in the 

deidentified sample on DRA-2 proficiency levels. There was a statistically significant difference in school 

attendance between Bridge and comparison students in the deidentified sample, such that Bridge 

students had higher levels of school attendance (p < .05). 

Summary: Confirmatory Questions 

The confirmatory questions focus on the impact of the Bridge program relative to the comparison group 

on change in reading proficiency. The analysis finds some evidence of beneficial effects of the Bridge 

program, particularly after the first year of programming. Results with the full and matched samples 
 Prepared by OMNI Institute 

Page 42

suggest that Bridge students generally become more likely to be proficient on the DRA-2 over time; in 

contrast, it appears that comparison students are either less likely to become proficient over time (as 

found for the full sample), or experience no change in their likelihood of being proficient (as with the 

matched sample). By their second and third years in the program, evidence suggests that Bridge 

students are outperforming their peers. 

Unfortunately, the deidentified data did not provide any further evidence in support of the Bridge 

program, finding no significant differences between Bridge students and their peers in reading 

proficiency. However, if it is expected that the high-risk Bridge sample would be doing worse over time 

than their peers at school without intervention, than the lack of significant differences between these 

groups could suggest that the programming helps Bridge students “keep up” with other students. It is 

important to note, however, that there were no baseline differences in DRA-2 proficiency, which limits 

this conclusion. Unfortunately, the data that could be collected for the deidentified sample, including 

just one year of data and no demographic information, limits the conclusions that can be drawn from 

these results. 

Overall, it does not appear that participation in the Bridge program is related to school attendance. For 

this secondary outcome, there is a consistent lack of differences among Bridge and comparison students 

across analytic models. Only the de-identified data show a significant impact of the program on school 

attendance. One factor that may play a role here is a lack of variability in the main sample; at baseline 

and all follow-ups, school attendance rates were consistently high among for both Bridge and 

comparison students. Statistically, this lack of variability makes it more difficult to detect significant 

effects. In practice, the Bridge program may not be able to impact school attendance rates among a 

sample that is generally attending school regularly (i.e., a ceiling effect). 

Exploratory Questions 

In the fourth year, data were sufficient to address the following exploratory research questions. 

1. Are more Bridge children reading at or above grade level by the end of third grade than 

comparison children? 

This exploratory research question was initially proposed to examine differences in 3rd grade reading 

levels between Bridge and comparison students using the Transitional Colorado Assessment Program 

(TCAP) measure, which at the time was Colorado’s standards-based assessment. However, over the 

course of the study the use of TCAP was discontinued; therefore, to address this question, we consider 

the only literacy measure available, the DRA-2, as the outcome measure. 

We examined mean differences in DRA proficiency between Bridge and comparison students for the 

follow-up period when they were in third grade. Overall, there was not a significant mean difference 

(t(126)=-.21, p = .84) in DRA proficiency between comparison students (M=.46) and Bridge students 

(M=.48) in third grade. Additionally, analyses using logistic regression with FIML indicated that there was 

not a significant difference in DRA proficiency between Bridge and comparison students in third grade.
 Prepared by OMNI Institute 

Page 43

2. Does the amount of programming predict reading gains? For example, does the number of years 

of participation in the program predict reading gains? 

To examine the benefit of additional years of participation in the project (i.e., dosage), we computed 

change scores for reading proficiency among Bridge students. The change score has values of -1 for 

decline, 0 for no change, and 1 for improvement. To reliably examine change over time, only Bridge 

students who remained in the program and have DRA proficiency data for at least three years were 

examined. This resulted in a sample of 39 Bridge students, representing just 11% of the total sample. Of 

these 39 Bridge students, the majority (62%; n= 24) were Kindergarten students at baseline (an 

additional six of these students were in 1st grade at baseline, six were in 2nd grade at baseline, and three 

were in 3rd grade at baseline). As shown in Table 19, in the first year the largest proportion of Bridge 

students became proficient (n=15, 38.5%), and the majority of these students were in Kindergarten 

when they entered Bridge (87%; 13 of 15). In the second year, the largest proportion of Bridge students 

remained proficient (n=15, 38.5%). In the third year, the largest proportion of students remained not 

proficient (n=16, 41%). These results suggest that the Bridge program has the largest impact in the first 

year of programming, which is not consistent with the results from analyses conducted with the full 

sample. Statistically, given the limited sample size here and the fact that the majority of this small 

sample were Kindergarten students at baseline, it may be that these findings are being driven by the 

fact that the pre-test measure for most of these students is the DRA-2 Word Analysis. It may also be 

that, in practice, it is easier to affect change in DRA-2 proficiency in the first year for students in earlier 

grades. 

TABLE 19 - DRA-2 PROFICIENCY CHANGE SCORES AMONG BRIDGE STUDENTS 

Change Scores (N = 39) 

Type of Change 

1st-Year Change 2nd-Year Change 3rd-Year Change 

n % n % n % 

Became Not Proficient 1 2.6% 13 33.3% 2 5.1% 

Remained Not Proficient 10 25.6% 10 25.6% 16 41.0% 

Remained Proficient 13 33.3% 15 38.5% 14 35.9% 

Became Proficient 15 38.5% 1 2.6% 7 17.9% 

To examine the benefit of amount of Bridges programming that students receive, we computed an 

aggregate dosage score that captures the number of different types of programming that students 

received in a given year. Scores included GR8 Readers books, Read Well, tutoring, and other non-literacy 

services (e.g., social-emotional programming, technology-based interventions); for example, Bridge 

students with a dosage score of one in year one received one out of four of these services in their first 

year in the program, and Bridge students with a dosage score of four in year one received all of these 

services in their first year of the program. 

As shown in Table 20, there is evidence of a dosage effect in the first and second years. For the 1-year 

follow-up, number of services was marginally related to DRA-2 proficiency (p = .07). For the 2-year 

follow-up, number of services was positively and significant related to DRA-2 proficiency (p < .001). For 

the 3-year follow-up, dosage was not significantly related to DRA-2 proficiency (p = .53), but the 
 Prepared by OMNI Institute 

Page 44

direction of the effect and the odds ratio were consistent with findings from the first two years. There 

were too few cases for reliable analysis of the 4-year follow-up (N = 19). 

TABLE 20 - EFFECTS OF NUMBER OF BRIDGE SERVICES ON DRA-2 PROFICIENCY 

DRA-2 Proficiency Logistic Regression

1-year follow-up 2-year follow-up 3-year follow-up

Predictors B OR Sig B OR Sig B OR Sig 

DRA-2 + WA Baseline 0.41 5.11 *** 0.42 6.37 *** 0.33 3.55 * 

Number of Bridge Services 0.13 1.20 * 0.33 1.52 *** 0.10 1.12 

N 226 144 50

*p< .10. **p < .05. ***p < .01.

3. What is the strength of the relationship between number of hours of participation and reading 

gains? Does this differ across program component? 

To address this question, we examined how the total number of hours of participation in Bridge was 

related to reading proficiency for Bridge students at each follow-up. As displayed in Table 21, there was 

a significant positive effect of hours of services received at the 1- and 2-year follow-ups (ps < .01). As 

noted previously, there were too few cases for reliable analysis of the 4-year follow-up (N = 19). 

TABLE 21 - EFFECTS OF BRIDGE HOURS OF SERVICES RECEIVED ON DRA-2 PROFICIENCY 

DRA-2 Proficiency Logistic Regression 

1-year follow-up 2-year follow-up 3-year follow-up

Predictors B OR Sig B OR Sig B OR Sig 

DRA-2 + WA Baseline 0.40 5.11 *** 0.43 6.38 *** 0.33 3.69 ** 

Total Hours of Bridge Services 0.21 1.02 *** 0.29 1.02 *** 0.12 1.01 

N 226 144 50

*p< .10. **p < .05. ***p < .01.

4. Which program activities are most closely associated to reading gains; what are the critical 

program elements for success? 

Prior to examining what program activities are most closely associated with reading gains, the number 

of GR8 Readers books received, hours in Read Well, hours of tutoring, and hours of other services 

received, correlations among the program activity dosages were examined within a given year. 

Correlations were examined for years 1-3, as once again, there were too few cases to examine the 4-

year follow-up (N = 19). Results, shown in Table 22, indicate that program activity dosages are highly 

correlated in each year, and seem to be most strongly correlated, on average, in the first year. 
 Prepared by OMNI Institute 

Page 45

Next, program activity dosages during a given year were examined as predictors of DRA proficiency 

among Bridge students. Results are displayed in Table 23. For the 1- and 2-year follow-ups, the number 

of GR8 Readers books had a statistically significant and positive effect on DRA-2 proficiency (ps < .05). 

Read Well, tutoring services, and other services had no independent relation to DRA-2 proficiency. 

When comparing the three core programming elements, the GR8 Readers program offers students the 

greatest agency, as students are allowed to select their own books to read, use in their tutoring sessions, 

and subsequently take home. It may be that this aspect of choice is consequential for engagement in 

reading, and, subsequently, proficiency. However, it may also be the case that students who are highly 

motivated to read are the ones who seek out more GR8 Readers books, and those students are also 

more likely to be proficient. 

TABLE 22 - WITHIN-YEAR CORRELATIONS AMONG BRIDGE PROGRAM ACTIVITY DOSAGE 

1 2 3 4 5 6 7 8 9 10 11 12

1. GR8 Readers – 1 yr - .69** .81** .65**

2. Read Well – 1 yr - .82** .93**

3. Tutoring – 1 yr - .83**

4. Other Services – 1 yr - 

5. GR8 Readers – 2 yr - .78** .76** .67** 

6. Read Well – 2 yr - .76** .87** 

7. Tutoring – 2 yr - .80** 

8. Other Services – 2 yr - 

9. GR8 Readers – 3 yr - .80** .80** .65** 

10. Read Well – 3 yr - .81** .86** 

11. Tutoring – 3 yr - .84** 

12. Other Services – 3 yr - 

**p < .01 

Additionally, the GR8 Readers program also has overlap with other programming, so results from these 

analyses should not be interpreted to suggest that GR8 Readers has a stand-alone effect. For example, 

tutors use the GR8 Readers books that students have chosen during their one-on-one sessions, and 

during the summer educators can choose to structure Read Well sessions to include GR8 Readers books. 

Also, the high correlations between GR8 Readers and the other literacy programming within years 

(ranging from .65 to .81) suggest that it is difficult to statistically parse out the unique impact of each 

literacy element. 

TABLE 23 - EFFECTS OF INDIVIDUAL BRIDGE PROGRAMS ON DRA-2 PROFICIENCY 

DRA-2 Proficiency Logistic Regression 

1-year follow-up 2-year follow-up 3-year follow-up

Predictors B OR Sig B OR Sig B OR Sig 

DRA-2 + WA Baseline 0.38 5.03 *** 0.42 6.54 *** 0.33 3.75 ** 

GR8Books 0.29 1.31 ** 0.36 1.36 ** 0.04 1.04 

Read Well -0.21 0.98 -0.10 0.99 0.20 1.02 

Tutoring -0.07 -.98 0.12 1.03 -0.06 0.99 

Other Services 0.30 1.01 0.02 1.00 -0.06 0.00 

N 226 144 50

*p < .10. **p < .05. ***p < .01.
 Prepared by OMNI Institute 

Page 46

5. Are results for grades K-3 maintained for Bridge children who reach fourth or fifth grade during 

the study period? Do these Bridge children show greater long-term gains in reading achievement 

than comparison children? 

To consider whether reading proficiency gains are maintained into the later grades among Bridge 

students, we examined change from baseline only among those students above grade three (see Table 

24). Unfortunately, the amount of data is limited for these students (Bridge n = 11, comparison n = 6); 

only the 2-year follow-up can be examined descriptively and should not be interpreted given the very 

low sample sizes. 

TABLE 24 - LONG-TERM DRA-2 PROFICIENCY CHANGE SCORES 

Long-Term Change Scores (Baseline to 2-Year Follow-Up) For Students in Fourth or Fifth Grade

Type of Change 

Bridge Students (N = 11) Comparison Students (N = 6) 

n % n % 

Became Not Proficient 1 9.1% 0 0.0% 

Remained Not Proficient 9 81.8% 4 66.6% 

Remained Proficient 1 9.1% 1 16.6% 

Became Proficient 0 0.0% 1 16.6% 

Summary: Exploratory Questions 

The exploratory questions focus first on the long-term effects of the Bridge program; namely, whether 

positive effects on literacy persist past third grade. Although the results suggest that there are no 

benefits to the Bridge program past third grade, it is important to note that the analyses are limited by 

the number of Bridge and comparison students for whom there is valid data at these later grades and 

should thus be interpreted with caution. It is also important to note that these analyses suggest that the 

greatest individual gains in proficiency are made during the first year of Bridge; however, this is also the 

period for which there are the most data, as missing data increases significantly at later follow-ups and 

as students progress into later grades. 

The exploratory questions also focus on understanding how the amount and type of Bridge 

programming is related to literacy. These results suggest that the number of Bridge programming 

elements that students receive (i.e., GR8 Readers, Read Well, tutoring, and other services) is positively 

related to DRA-2 proficiency; further, the number of hours of programming across these services is also 

positively related to DRA-2 proficiency. Specifically, it appears that GR8 Readers may be a particularly 

effective program element. However, selection bias may be affecting these dosage results. It may be 

that the greater motivation of those who sought the most program elements and hours rather than the 

program itself affected the results. Additionally, GR8 Readers books are incorporated into both Read 

Well and tutoring sessions, are highly correlated with these programming elements, and thus the effect 

of this unique aspect of programming should be interpreted with caution. 
 Prepared by OMNI Institute 

Page 47

Study Limitations

Program implementation and evaluation activities during the fourth year of the Social Innovation Fund 

Implementation and Impact Study yielded a number of important lessons for both OMNI and the Bridge 

Project. 

Program Impacts 

Over time, Bridge students appear to fare better in DRA-2 proficiency than their peers who also live in 

Denver public housing. On average, students who are not involved with the Bridge Project’s literacy 

programming appear to fall behind, as they become less likely to score proficient on the DRA-2; 

meanwhile, students who are involved show desired growth, as they become more likely to score 

proficient on the DRA-2. School attendance rates, a secondary outcome of interest, do not appear to be 

impacted by the Bridge Project. Although the study demonstrates some positive findings, there were 

several challenges and limitations to this study. 

Program Fidelity and Variation 

Read Well and tutoring session observations indicate improvements from the previous academic year in 

several areas including greater consistency in program delivery. Over the course of the five years, each 

year the program has improved. The Bridge Project has taken several steps over the course of the five 

years to improve the quality and consistency of the program. In the 2015-2016 school year, Bridge 

educators began using education interns and student teachers. This allowed for the presence of a 

second adult in the room, which allowed for more teaching time and reduced behavioral problems. The 

Bridge Project also made some staffing changes to create a position to oversee the education program 

at each of the sites. This allows for more support and training directly to the educators. 

To improve the tutoring program, the Bridge Project hired a full time, on-site volunteer coordinator. This 

new position will be responsible for ongoing training and evaluation throughout the year. The hope is 

that with on-site support, the volunteers will be able to better support the literacy programming. 

Comparison Student Recruitment 

To address low recruitment early in the project, the study hired a part-time data coordinator in 2014 

focused solely on comparison study recruitment. The data coordinator has been active in the two 

comparison site housing communities, recruiting at neighborhood events, and schools. Over the course 

of the study, the research team implemented several strategies with the aim of improving recruitment 

efforts, including: 

• Attending events at each of the comparison neighborhood schools, such as family literacy night 

• Meetings with the Denver Housing Authority liaisons to coordinate on-site events 

• Passing out flyers in the neighborhoods inviting families to a meet and greet event 
 Prepared by OMNI Institute 

Page 48

• On-site meet and greet event with refreshments and gift card incentives 

• Coordinating with the field placement program at University of Denver School of Social Work 

field placement program to coordinate efforts with students working in these communities 

• Attending the weekly Foodbank event to meet families 

• Meeting with families at the youth activities program in Sun Valley 

Despite these increased efforts, there still remained some barriers. One of the main challenges was 

access to information. When the researcher attended the neighborhood events, parents were not 

prepared with the needed information to enroll in the study, such as DPS ID number. When the data 

coordinator would try and follow-up with them to get the additional information, often the phone 

numbers provided no longer worked or voice mail messages were not returned. 

A second challenge had to do with language. The Denver public housing communities are quite diverse 

and speak a number of different languages. The research coordinator was able to provide information 

and had forms available in both Spanish and English but was not able to provide information in any 

other languages. The limits of this may have excluded children from participating in the study. 

Missing Data 

In addition to difficulties recruiting comparison group participants, the evaluation has also faced 

challenges resulting from missing literacy outcome data. The research team has improved their data 

collection processes by reviewing intake forms as they come in and initiating contact with Bridge staff or 

parents as soon as possible to attempt to obtain or correct missing school ID numbers. The team, in 

collaboration with the Bridge staff, has conducted regular audits to evaluate why data may be missing. 

In addition to improving data collection methods, the research team has been reviewing internal 

processes for how data is managed. Several internal steps have been taken to ensure that missing data 

is not the result of problems within the data file and has instituted a variety of processes related to 

internal auditing of the data file to correct for any problems that may cause cases to be excluded. This 

auditing process has improved the number of cases that may have been left out due to a technical or 

syntax error rather than true missing data. These processes have improved the match rate, with 71% of 

the enrolled participants matched to literacy outcome data, as compared to a 63% match rate in the 

previous year. There were also attempts made to address the missing data statistically. However, given 

the low sample sizes, there was not enough power to use imputation methods. Instead, full information 

maximum likelihood estimation methods were used in analyses, which proved to be an effective 

approach given the ability to reliably analyze data from the 3-year follow-up, and the statistically 

significant findings that emerged. 
 Prepared by OMNI Institute 

Page 49

Summary and Conclusions

Since 2012, this longitudinal study has examined the dosage, fidelity, and impact of the Bridge Project’s 

early literacy interventions. Over the course of five years of programming, and as a direct result of the 

Social Innovation Fund, the capacity of the Bridge project to provide high quality programming to 

students has increased. Furthermore, results from the quasi-experimental, between-group study yielded 

positive results, increasing the level of evidence for the Bridge Project’s multi-pronged, early literacy 

programming, designed to increase low-income students’ reading proficiency. 

Specifically, when compared with a matched sample of their peers (i.e., other students who were similar 

across a variety of socioeconomic and demographic indicators, and also similarly residing in public 

housing), there were no pre-existing differences between Bridge and non-Bridge students in their 

likelihood of scoring proficient on the key indicator of literacy, the DRA-2; however, at the end of their 

second and third years with the program, Bridge students were more likely to be proficient than nonBridge students. Based on the patterns of change demonstrated within each group, it appears that while 

Bridge students demonstrate growth in proficiency over time, non-Bridge students remain stable, and 

Bridge students ultimately outpace their peers who are not receiving the additional Bridge literacy 

interventions. 

In addition to increasing the evidence-base for Bridge’s early literacy intervention, SIF funding supported 

an implementation evaluation designed to identify opportunities to strengthen programming. Based on 

observation findings each year, the evaluation team offered suggestions aimed at increasing consistency 

in program implementation, enhancing staff and volunteer training/development opportunities, and 

creating stronger alignment between individual tutor skills and student needs. In response, substantial 

enhancements were made. First, the Bridge Project added a Director of Volunteers and Tutoring, who 

provides on-site orientations to help volunteers acclimate to the tutoring role, visits sites weekly on a 

rotating schedule to provide hands-on support to tutors as they are working with students, and 

coordinates a Volunteer Leadership Committee to support tutor retention. Second, the Bridge Project 

hired an Education Director, who is charged with observations and coaching around best practices to 

ensure fidelity within all the educational programming. Third, an observational tool was developed and 

refined throughout the SIF study. This tool is now used for peer observations so educators can receive 

feedback and learn from and collaborate with other educators delivering the Read Well curriculum. 

Finally, the Bridge Project instituted the YPQI process in order to receive more continual feedback 

amongst its staff and volunteers on the quality of programming being delivered. By receiving annual 

feedback on the programming and its impact on student outcomes, the Bridge Project was able to adjust 

and consider programmatic changes to enhance the literacy experiences of its students. 

Further, dosage results demonstrate that over the years of the study, the Bridge Project was able to 

generally increase the amount of early literacy provided to study participants. Specifically, the SIFfunded Bridge Project study reached a total of 389 students with literacy services since 2012 and 

steadily increased the average number of Read Well and tutoring sessions that students received each 
 Prepared by OMNI Institute 

Page 50

year. In addition, participation in GR8 Readers increased over the course of the study, and in 2015-16, 

the program returned to an average of five books per students. 

The mission of the Bridge Project is to provide educational opportunities to children living in Denver’s 

public housing neighborhoods in order to increase high school graduation and college attendance or 

post-secondary vocational training. The Bridge Project has developed a multi-pronged early literacy 

intervention designed to improve reading proficiency that consists of a combination of three direct 

literacy programs (Read Well, tutoring, and GR8 Readers). The current study identified opportunities to 

enhance program infrastructure and implementation, including adding and refining staff roles, 

strengthening its volunteer training program, and implementing standardized continuous quality 

improvement efforts. Importantly, study results are supportive of ‘moderate’ evidence that the Bridge 

Project early literacy intervention increases reading proficiency for students residing in or near public 

housing communities. As noted by the Annie E. Casey Foundation in 2010, third-grade reading 

proficiency is critical for academic success, high school graduation, and breaking the cycle of 

intergenerational poverty. Study findings suggest that the Bridge Project’s early literacy intervention is 

effective at improving reading proficiency for children living in poverty. 
Prepared by OMNI Institute 

Page 51

References

Child Trends Data Bank (2012a, April). Parental Education: Indicators on children and youth. Accessed 

online at http://www.childtrendsdatabank.org/alphalist?q=node/183. 

Child Trends Data Bank (2012b, July). Parental Expectations for their children’s academic attainment:

Indicators on children and youth. Accessed online at 

http://www.childtrendsdatabank.org/alphalist?q=node/366. 

Hemphill, F.C., and Vanneman, A. (2011). Achievement Gaps: How Hispanic and White Students in Public 

Schools Perform in Mathematics and Reading on the National Assessment of Educational 

Progress (NCES 2011-459). National Center for Education Statistics, Institute of Education 

Sciences, U.S. Department of Education. Washington, DC 

Pearson Education, Inc. (2011). K-8 technical manual: Developmental Reading Assessment. 2nd Ed. 

Saddle River, NJ: Pearson Education, Inc. 

Rathvon, N. (2008). Effective school interventions: Evidence based strategies for improving student 

outcomes. 2nd Ed. New York: Guilford Press. 

Vanneman, A., Hamilton, L., Baldwin Anderson, J., and Rahman, T. (2009). Achievement Gaps: How Black 

and White Students in Public Schools Perform in Mathematics and Reading on the National 

Assessment of Educational Progress, (NCES 2009-455). National Center for Education Statistics, 

Institute of Education Sciences, U.S. Department of Education. Washington, DC. 
 Prepared by OMNI Institute 

Page 52

Appendix A – Read Well Observation Tool

Session Date:_______________ Observer: _______________ Start Time:__________ End 

Time:___________ 

Educator Name: __________________________ RW Unit #: _____ RW Book: 

_________________________ 

# Kids at start:________ # Kids at end:_______ # Kids total:________ (____boys/____girls) 

# Kids matched with a tutor: _______ (Observer will need to request this from the Site 

Manager/Educator) 

Site: Columbine Westwood South Lincoln Quigg Newton 

Location in site (e.g., computer room):____________________ Grade Level: K/1 2/3

Environment

Site has designated Read Well space Yes No

Materials have a designated place Yes No

Visual aids on walls Yes No

External Distractions None Some Many 

Internal Distractions None Some Many 

Notes: 

Educator and Other Adults

Lesson and materials planned? Yes No

Content and level generally appropriate? Yes No

Educator uses innovative teaching strategies? Yes No

Pacing of the session for the group overall Too fast About right Too slow 

Did the Educator divide students into sub-groups during the session? Yes No

If so, number of groups: ______ 

Were all groups supervised by either an educator or a volunteer? Yes No

Did adults other than the Read Well 

Educator enter the session? 

 Yes No

(If yes) Number of other adults 

entering session: _______ 

Time Purpose Distracting 

1. Yes No

2. Yes No

3. Yes No

4. Yes No

(If applicable) Other adults in class are 

helpful. 

 Never Rarely Sometimes Often 
 Prepared by OMNI Institute 

Page 53

(If applicable) Other adults in class are 

disruptive. 

 Never Rarely Sometimes Often 

Educator’s overall instructional skill level Highly skilled Average Limited/Needs guidance 

Notes: 

Students

None Some Most All N/A

Students engaged with material 

Students follow educator instructions 

Students demonstrate learning of literacy skills 

Students responsive to incentives 

Students responsive to disincentives 

Students display knowledge gaps 

Students demonstrate behavioral issues 

Students eagerly help educator (e.g., pick up pencils, 

help classmates, distribute books) 

Number of students entering session 

late: ________ 

Number of students leaving session 

early: ________ 

Time Coming or Going Disruptive? 

1. Coming Going Yes No

2. Coming Going Yes No

3. Coming Going Yes No

4. Coming Going Yes No

Notes: 

Classroom Management 

Never Rarely Sometimes Often N/A

Effective use of discipline (e.g., use of reflection forms) 

Effective use of rewards (e.g., PBIS tickets) 

Efficient transitions 

Attentive to needs of students 

Provides remediation when necessary 

Provides encouragement 

Provides clear instructions 

Notes (including count of a) reflection forms used/observed and/or b) PBIS tickets distributed): 

Note: N/A is intended to indicate that an Educator did not use discipline/rewards/remediation.
 Prepared by OMNI Institute 

Page 54

 Session Activity Group # (if applicable): _______

Type of activity: Letter Sound Cards Decoding Sheets Story Reading Follow-up Activity 

 Other (describe): 

Activity start time: _________ End time: _________ Number of children in group:_____ 

Number of children entering during activity:_____ Number of children leaving during activity: _____ 

Children who seem comfortable with the material: None Some Most All 

Children who seem somewhat comfortable with the material: None Some Most All 

Children who seem uncomfortable with the material: None Some Most All 

Activity overseen by: 

 Educator Intern Volunteer Educator, while working with other group Not overseen 

Reading components addressed during activity: 

 Comprehension Fluency Phonics Vocabulary Phonemic awareness 

Activity included (check all that apply): 

 Verbal participation from individual students Verbal participation from groups of students 

 Individual work Group work Literacy-focused tasks Drawing/coloring Writing Reading 

 Educator sourced materials/tasks Read Well curriculum materials/tasks 

Notes: 

 Session Activity Group # (if applicable): _______

Type of activity: Letter Sound Cards Decoding Sheets Story Reading Follow-up Activity 

 Other (describe): 

Activity start time: End time: Number of children in group:_____ 

Number of children entering during activity:_____ Number of children leaving during activity: _____

Children who seem comfortable with the material: None Some Most All 

Children who seem somewhat comfortable with the material: None Some Most All 

Children who seem uncomfortable with the material: None Some Most All 

Activity overseen by: 

Educator Intern Volunteer Educator, while working with other group Not overseen 

Reading components addressed during activity: 

 Comprehension Fluency Phonics Vocabulary Phonemic awareness 

Activity included (check all that apply): 

Verbal participation from individual students Verbal participation from groups of students 

Individual work Group work Literacy-focused tasks Drawing/coloring Writing Reading 

Educator sourced materials/tasks Read Well curriculum materials/tasks 

Notes: 
 Prepared by OMNI Institute 

Page 55

Additional Observer Notes 
 Prepared by OMNI Institute 

Page 56

Appendix B – Tutoring Observation Tool 

Session Date:_______________ Observer: _______________ Start Time:__________ End 

Time:___________ 

Site: Columbine Westwood South Lincoln Quigg Newton 

Location in site (e.g., computer room):_________________ Length of tutoring relationship (in 

yrs):________ 

Tutor name:_______________________ Tutor Gender: Male Female

Student Name:_____________________________ Grade:________ Student Gender: Male 

Female 

Was a language other than English spoken during literacy components of the session? 

 Yes No

Environment

Site has a designated Tutoring space Yes No Materials have a place Yes No

External Distractions None Some Many 

Internal Distractions None Some Many 

Session Information

 Activity covered this session? Start 

Time 

End 

Time 

Homework/Activity Time Yes 

No

 Homework subjects covered: 

 Math LA/Reading Science Social Studies 

 Other:__________________ 

Skill Building Activity Yes 

No

Describe: 

GR8 Reader book reading Yes 

No

 Book:_____________________________________ 

GR8 Reader packet Yes 

No

 Reading components 

covered: 

Extensively Briefly Not 

Covered 

Phonics 

Phonemic Awareness 

Vocabulary 
 Prepared by OMNI Institute 

Page 57

Fluency 

Comprehension 

Reward time Yes 

No

Briefly describe: 

Other (notes): 

Tutor 

Lesson and materials planned? Yes No

Content and level generally 

appropriate? 

 Yes No

Transitions generally 

smooth/efficient? 

 Yes No

Pacing of the session Too fast About right Too 

slow 

Interactions with students Mostly negative Neutral 

Mostly positive 

Tutoring style Mostly disengaged Neutral 

Mostly engaged 

Use of rewards/incentives (e.g., 

stickers) 

No rewards Some rewards 

Many rewards

Use of disincentives (e.g., 

warning) 

 No disincentives Some disincentives Many 

disincentives 

Behavior management/control of 

lesson 

 Limited control Moderate control Strong 

control 

Tutor’s overall instructional skill 

level 

 Limited/Needs guidance Average Highly 

skilled 

Notes: 

Student 

Attention to the tutor Mostly unfocused Sometimes focused Mostly 

focused 

Engagement with the material Mostly disengaged Sometimes engaged 

Mostly engaged 
 Prepared by OMNI Institute 

Page 58

Demonstrations of learning None Some Many 

Follows tutor’s instructions Never Rarely Sometimes Usually 

Response to 

rewards/incentives 

 Negative response Neutral/No response Positive response 

N/A (not used) 

Response to disincentives Negative response Neutral/No response Positive response 

N/A (not used)

Knowledge gaps No gaps Few gaps Some gaps 

Many gaps 

Behavioral issues No issues Few issues Some issues 

Many issues 

Notes (including reasons for distractions, e.g., other tutor pairs in the room):

Additional Observer Notes 
 Prepared by OMNI Institute 

Page 59

Appendix C – Descriptive Study Participant Data 

Tables

Family Level Demographic Data

FAMILY SIZE 

Numeric Code Bridge Comparison Variable Type 

2 N/A 12 (4.6%) 8 (7.4%) Ratio 

3 N/A 33 (12.5%) 18 (16.7%) 

4 N/A 68 (25.9%) 21 (19.4%) 

5 N/A 58 (22.1%) 21 (19.4%) 

6 N/A 42 (16.0%) 11 (10.2%) 

7 N/A 9 (3.4%) 8 (7.4%) 

8 N/A 18 (6.8%) 5 (4.6%) 

9 to 13 N/A 23 (8.8%) 7 (6.5%) 

Missing -9 N/A 9 (8.3%) 

Total 263 102 

NUMBER OF CHILDREN IN HOUSEHOLD 

Numeric Code Bridge Comparison Variable Type 

1 N/A 29 (11.0%) 13 (12.0%) Ratio 

2 N/A 53 (20.2%) 19 (17.6%) 

3 N/A 77 (29.3%) 26 (24.1%) 

4 N/A 53 (20.2%) 15 (13.9%) 

5 N/A 14 (5.3%) 9 (8.3%) 

6 N/A 15 (5.7%) 6 (5.6%) 

7 N/A 11 (4.2%) 4 (3.7%) 

8 or 9 N/A 9 (3.4%) 4 (3.7%) 

Missing -9 2 (0.8%) 12 (11.1%) 

Total 263 108 

FAMILY MOBILITY: HOW MANY TIMES HAS THE FAMILY MOVED FROM ONE HOUSEHOLD 

TO ANOTHER IN THE PAST 3 YEARS? 

Numeric Code Bridge Comparison Variable Type 

0 times N/A 123 (46.8%) 40 (37.0%) Ratio 

1 time N/A 87 (33.1%) 24 (22.2%) 

2 times N/A 31 (11.8%) 11 (10.2%) 

3 times N/A 8 (3.0%) 7 (6.5%) 

4 times N/A 4 (1.5%) 6 (5.6%) 

5 times N/A 0 (0.0%) 2 (1.9%) 

6 times N/A 0 (0.0%) 1 (0.9%) 

8 times N/A 0 (0.0%) 4 (3.7%) 

10 times N/A 1 (0.4%) 0 (0.0%) 

Missing -9 9 (3.4%) 13 (12.0%) 
 Prepared by OMNI Institute 

Page 60

Total 263 108

FAMILY INCOME STATUS 

Numeric Code Bridge Comparison Variable Type 

Less than $10,000 0 150 (57.0%) 77 (71.3%) Interval 

Between $10,000 and 24,500 1 77 (29.3%) 12 (11.1%) 

Between 25,000 and $49,999 2 30 (11.4%) 8 (7.4%) 

Between $50,000 and $74,999 3 1 (0.4%) 0 (0.0%) 

$75,000 or more 4 3 (1.1%) 1 (0.9%) 

Missing -9 2 (0.8%) 10 (9.3%) 

Total 263 108

LANGUAGE SPOKEN IN THE HOME 

Numeric Code Bridge Comparison Variable Type 

English 1 156 (59.3%) 87 (80.6%) Nominal 

Spanish 2 20 (7.6%) 2 (1.9%)

Vietnamese 3 17 (6.5%) 0 (0.0%)

Other 4 47 (17.9%) 6 (5.6%) 

Missing -9 23 (8.8%) 13 (12.0%) 

Total 263 108

PARENTAL EDUCATIONAL LEVEL 

Numeric 

Code 

Bridge Comparison Variable 

Type 

No schooling completed 0 43 (16.3%) 5 (4.6%) Ordinal 

Some school (but less than diploma or GED) 1 72 (27.4%) 33 (30.6%)

High School Graduate/ GED 2 55 (20.9%) 29 (26.9%)

Some college or technical school 3 57 (21.7%) 30 (27.8%)

College Degree 4 32 (12.2%) 9 (8.3%)

Post College Degree 5 1 (0.4%) 0 (0.0%)

Missing -9 3 (1.1%) 2 (1.9%) 

Total 263 108

Child Level Demographic Data

GENDER 

Numeric Code Bridge Comparison Variable Type 

Male 0 137 (52.1%) 54 (50.0%) Binary 

Female 1 126 (47.9%) 54 (50.0%) 

Total 263 108

DID THE CHILD ATTEND PRESCHOOL? 

Numeric Code Bridge Comparison Variable Type 

No 0 77 (29.3%) 32 (29.6%) Binary 

Yes 1 175 (66.5%) 43 (39.8%) 

Missing -9 11 (4.2%) 33 (30.6%) 

Total 263 108 
 Prepared by OMNI Institute 

Page 61

RACE OF CHILD*

Numeric Code Bridge Comparison Variable Type 

Black/African-American N/A 87 (36.6%) 43 (44.8%) Binary 

Asian N/A 32 (13.4%) 4 (4.2%) 

White/Caucasian N/A 47 (19.7%) 19 (19.8%) 

Hispanic or Latino N/A 118 (47.2%) 49 (48.0%) 

American Indian or Native American N/A 6 (2.5%) 9 (9.4%) 

Multiracial N/A 11 (5.4%) 9 (11.1%) 

Unknown N/A 5 (2.5%) 0 (0.0%)

Other N/A 48 (20.2.0%) 5 (4.9%) 

Total 263 108 

 * Some parents identified their child as belonging to multiple racial categories. As such, totals sum to greater than 

the total number of students in each group and proportions sum to greater than 100%. Each racial category is 

coded as a separate categorical variable coded 1”yes”, 0 ”no”, -9 ”missing”.

CHILD GRADE 

Numeric Code Bridge Comparison Variable Type 

Kindergarten 0 124 (47.1%) 29 (26.9%) Ordinal 

1

st Grade 1 49 (18.6%) 26 (24.1%)

2

nd Grade 2 47 (17.9%) 27 (25.0%)

3

rd Grade 3 43 (16.3%) 26 (24.1%)

Total 263 108

Home Literary Environment 

FREQUENCY OF READING WITH CHILD IN PAST WEEK 

Numeric Code Bridge Comparison Variable Type 

Not at all 0 24 (9.1%) 6 (5.6%) Ordinal 

Once or twice 1 78 (29.7%) 17 (15.7%) 

Three or more times 2 81 (30.8%) 34 (31.5%) 

Every day 3 78 (29.7%) 49 (45.4%) 

Missing -9 2 (0.8%) 2 (1.9%) 

Total 263 108 

FREQUENCY OF FAMILY DINNERS 

Numeric Code Bridge Comparison Variable Type 

Not at all 0 4 (1.5%) 6 (5.6%) Ordinal 

Once or twice 1 37 (14.1%) 7 (6.5%) 

Three or four times 2 50 (19.0%) 28 (25.9%) 

Five or more times 3 168 (63.9%) 59 (54.6%) 

Missing -9 4 (1.5%) 8 (7.4%) 

Total 263 108 
 Prepared by OMNI Institute 

Page 62

BOOKS OWNED BY STUDY CHILDREN 

Numeric Code Bridge Comparison Variable Type 

10 or fewer books N/A 100 (38.0%) 28 (25.9%) Ratio 

11 to 50 books N/A 118 (44.9%) 28 (25.9%) 

More than 50 books N/A 35 (13.3%) 29 (26.9%) 

Missing N/A 10 (3.8%) 23 (21.3%) 

Total 263 108

Parental Expectations and Aspirations

CHILDREN HOW FAR DO YOU WANT YOUR CHILD TO GO IN FORMAL SCHOOLING? 

Numeric Code Bridge Comparison Variable Type 

Finish Elementary School 0 3 (1.1%) 8 (7.4%) Ordinal 

Finish Middle School 1 0 (0.0%) 1 (0.9%) 

Finish High School 2 11 (4.2%) 9 (8.3%) 

Attend Trade School 3 3 (1.1%) 0 (0.0%) 

Attend College or University 4 21 (8.0%) 23 (21.3%) 

Finish College or University 5 219 (83.3%) 60 (55.6%) 

Missing -9 6 (2.3%) 7 (6.5%) 

Total 263 108 

SCHOOLING HOW FAR DO YOU THINK YOUR CHILD WILL GO IN FORMAL SCHOOLING? 

Numeric Code Bridge Comparison Variable Type 

Finish Elementary School 0 0 (0.0%) 0 (0.0%) Ordinal 

Finish Middle School 1 0 (0.0%) 1 (0.9%) 

Finish High School 2 10 (3.8%) 15 (13.9%) 

Attend Trade School 3 4 (1.5%) 3 (2.8%) 

Attend College or University 4 31 (11.8%) 10 (10.2%) 

Finish College or University 5 209 (79.5%) 54 (53.7%) 

Missing -9 9 (3.4%) 19 (18.5%) 

Total 263 108 
 Prepared by OMNI Institute 

Page 63

Appendix D – Bridge Study Student Intake Forms

Parent/Guardian Name: 

Address: City: Zip Code: 

Home Phone: Cell Phone: Work Phone: Email: 

Where do you live? 

 Lincoln Park/La Alma (North or South 

Lincoln) 

 Westridge 

 Columbine  Sun Valley 

 Westwood  Quigg Newton 

 Other DHA housing (Please specify): ________________________

 Live in non-DHA housing

What language does your family speak in the home most often?

 English  Spanish  Vietnamese  Other: _________________ 

In the past 3 years, how many times has your family moved from one 

home or household to another? 

How many family members live in your household? 

How many children (0-18 years old) live in your household? 

In the past week, how many times has most or all of your family eaten dinner together, 

either at home or

somewhere else?  Not at all  Once or 

twice 3 or 4 times 5 or more times

What is the highest grade or year of school of 

any adult in your household? 

What is the annual income from all 

sources for your family? 

 No schooling completed 

 Some school (but less than high school 

diploma or GED) 

 High School Graduate/GED 

 Some college or technical school 

 College degree 

 Post college degree 

 Less than $10,000 

 Between $10,000 and $24,999 

 Between $25,000 and $49,999 

 Between $50,000 and $74,999 

 $75,000 or more 

How many people, including you, depend 

on this income? ________ 

Does your family qualify for free or reduced-price lunch at school? (check one) 

 Free lunch  Reduced-price lunch  No, we do not 

qualify.
 Prepared by OMNI Institute 

Page 64

For Office Use Only: Please complete this section if at least one child is in Kindergarten, 1st, 2nd, or 3rd grade 

Date Form Completed: _________________ Received by (name): __________________________________ 

Bridge Site:  Columbine  Lincoln Park  Quigg Newton  Westwood 

Bridge staff - Please enter the Bridge IDs for each of the children listed on this form. 

Child 1: Child 2: Child 3: Child 4: Child 5: Child 6: 

Staff –Please return the Participant Information Form to the DU drop box at your site. 

Child 1 

Child Name Date of Birth Age 

Gender 

Male

 Female School School Student ID# Grade 

Is your child (please check only one)?

Is your child of Hispanic, 

Latino or Spanish origin? 

 Yes

 No

 White/Caucasian 

 Black/African American 

 Asian 

 American Indian or Alaska Native 

 Native Hawaiian/Pacific Islander 

 Multiracial 

 Unknown 

 Other (please specify): 

_________________________ 

Did your child attend preschool?  No  Yes. If yes, where_____________________ 

Has your child participated at Bridge before this school year (2013-14)?  No  Yes

How far do you want your child to go in formal 

schooling? 

 Finish elementary school 

 Finish middle school 

 Finish high school 

 Attend trade school 

 Attend college or university 

 Finish college or university 

How far do you think your child will go in formal 

schooling? 

 Finish elementary school 

 Finish middle school 

 Finish high school 

 Attend trade school 

 Attend college or university 

 Finish college or university 

About how many books does your child have of his or her own (including those shared with brothers or 

sisters)? ___________ 

How many times have you or someone in your family read to your child in the past week? 

 Not at all  Once or twice  3 or more times  Everyday 

Child 2 

Child Name Date of Birth Age Gender 

Male

 Female

School School Student ID# Grade 

Is your child (please check only one)? Is your child of Hispanic, 

Latino or Spanish origin? 

 Yes

 No

 White/Caucasian 

 Black/African American 

 Asian 

 American Indian or Alaska Native 

 Native Hawaiian/Pacific Islander 

 Multiracial 

 Unknown 

 Other (please specify) 

Did your child attend preschool?  No  Yes. If yes, where_____________________ 

Has your child participated at Bridge before this school year (2013-14)?  No  Yes
 Prepared by OMNI Institute 

Page 65

How far do you want your child to go in formal 

schooling? 

 Finish elementary school 

 Finish middle school 

 Finish high school 

 Attend trade school 

 Attend college or university 

 Finish college or university 

How far do you think your child will go in formal 

schooling? 

 Finish elementary school 

 Finish middle school 

 Finish high school 

 Attend trade school 

 Attend college or university 

 Finish college or university 

About how many books does your child have of his or her own (including those shared with brothers or 

sisters)? ___________ 

How many times have you or someone in your family read to your child in the past week? 

 Not at all  Once or twice  3 or more times  Everyday 
 Prepared by OMNI Institute 

Page 66

Appendix E – Comparison Student Intake Forms

Parent/Guardian Name: 

Address: City: Zip Code: 

Home Phone: Cell Phone: Work Phone: Email: 

Where do you live? 

Where do you live? 

 Lincoln Park/La Alma (North or South 

Lincoln) 

 Westridge 

 Columbine  Sun Valley 

 Westwood  Quigg Newton 

 Other DHA housing (Please specify): ________________________

 Live in non-DHA housing 

What language does your family speak in the home most often?

 English  Spanish  Vietnamese  Other: _________________ 

In the past 3 years, how many times has your family moved from one 

home or household to another? 

How many family members live in your household? 

How many children (0-18 years old) live in your household? 

In the past week, how many times has most or all of your family eaten dinner together, 

either at home or

somewhere else? 

 Not at all

 Once or 

twice 3 or 4 times 5 or more times

What is the highest grade or year of school of 

any adult in your household? 

What is the annual income from all 

sources for your family? 

 No schooling completed 

 Some school (but less than high school 

diploma or GED) 

 High School Graduate/GED 

 Some college or technical school 

 College degree 

 Post college degree 

 Less than $10,000 

 Between $10,000 and $24,999 

 Between $25,000 and $49,999 

 Between $50,000 and $74,999 

 $75,000 or more 

How many people, including you, depend 

on this income? ________ 

Does your family qualify for free or reduced-price lunch at school? (check one) 

 Free lunch  Reduced-price lunch  No, we do not 

qualify.
 Prepared by OMNI Institute 

Page 67

For Office Use Only 

Date Form Completed: _________________________

Received by (name): __________________________________ 

Study ID (to be entered by OMNI): ______________

Child 1

Child Name Date of Birth Age Gender 

Male

 Female School School Student ID# Grade 

Is your child (please check only one)? Is your child of Hispanic, 

Latino or Spanish origin? 

 Yes

 No

 White/Caucasian 

 Black/African American 

 Asian 

 American Indian or Alaska Native 

 Native Hawaiian/Pacific Islander 

 Multiracial 

 Unknown 

 Other (please specify) 

Did your child attend preschool?  No  Yes. If yes, where_____________________

How far do you want your child to go in formal 

schooling? 

 Finish elementary school 

 Finish middle school 

 Finish high school 

 Attend trade school 

 Attend college or university 

 Finish college or university 

How far do you think your child will go in formal 

schooling? 

 Finish elementary school 

 Finish middle school 

 Finish high school 

 Attend trade school 

 Attend college or university 

 Finish college or university 

About how many books does your child have of his or her own (including those shared with brothers or 

sisters)? ___________ 

How many times have you or someone in your family read to your child in the past week? 

 Not at all  Once or twice  3 or more times  Everyday 

Child

2

Child Name Date of Birth Age 

Gender 

Male

 Female

School School Student ID# Grade 

Is your child (please check only one)? Is your child of Hispanic, 

Latino or Spanish origin? 

 Yes

 No

 White/Caucasian 

 Black/African American 

 Asian 

 American Indian or Alaska Native 

 Native Hawaiian/Pacific Islander 

 Multiracial 

 Unknown 

 Other (please specify) 

Did your child attend preschool?  No  Yes. If yes, where_____________________
 Prepared by OMNI Institute 

Page 68

How far do you want your child to go in formal 

schooling? 

 Finish elementary school 

 Finish middle school 

 Finish high school 

 Attend trade school 

 Attend college or university 

 Finish college or university 

How far do you think your child will go in formal 

schooling? 

 Finish elementary school 

 Finish middle school 

 Finish high school 

 Attend trade school 

 Attend college or university 

 Finish college or university 

About how many books does your child have of his or her own (including those shared with brothers or 

sisters)? ___________ 

How many times have you or someone in your family read to your child in the past week? 

 Not at all  Once or twice  3 or more times  Everyday