IMPACT OF NEBRASKA READING FIRST ON

**READING COMPREHENSION**

ACROSS THREE YEARS OF IMPLEMENTATION

2005-2007

Prepared by

Malinda Murphy Ph.D.

Guy Trainin Ph.D.

![](media/image1.jpeg){width="1.520138888888889in"
height="0.6215277777777778in"}

![](media/image2.wmf){width="1.2361111111111112in"
height="0.5069444444444444in"}

![](media/image2.wmf){width="1.2361111111111112in"
height="0.5069444444444444in"}

![](media/image3.jpeg){width="5.1402777777777775in"
height="0.5902777777777778in"}

OVERVIEW

The current report focuses on the impact of Reading First on reading
comprehension across the first three years of implementation in
Nebraska. Grade level performance is defined as performance at or above
the 40^th^ percentile using grade level standards. Within each grade
level section reading comprehension achievement is shown for all
students and then by cluster membership. As stated in the 2006-2007
annual report, clusters are used to group similar schools together and
enable Reading First leadership and school administration to make valid
comparisons. Given the significant differences between schools in size,
proportion of minority, disadvantaged and special education students, it
was clear that analyzing student achievement based on cluster membership
would better allow educators to make meaningful comparisons to school
facing similar challenges and thus make instructional adjustments that
meet the unique needs of students in their schools. Cluster membership
presented in this report is consistent with those reported in the annual
report. Achievement gaps between English language learners (ELL),
economically disadvantaged students (FRL), minority students, and
student with disabilities (special education) and their peers are
presented within each grade level section as results vary between grades
and by category.

FIRST GRADE

Each spring a random sample of first grade students throughout Reading
First schools in Nebraska are selected to take the Gray Oral Reading
Test (GORT). The GORT is an individually administrated norm referenced
test. Nebraska law does not allow group administered tests before second
grade. Four students (and two alternates) were randomly selected to
complete this assessment from each first grade classroom in Reading
First schools. This sampling procedure increases the relative impact of
smaller classrooms as compared to measures in second and third grade.

The GORT comprehension subtest was selected for administration due to
its high level of reliability. Reliability estimates indicates "the
extent to which individual differences in test scores are attributable
to 'true' differences in the characteristics under consideration and the
extent to which they are attributable to chance errors" (Anastasi &
Urbina, 1997, p. 84). The reliability coefficients for the GORT
comprehension subtest for the age range of this sample ranges between
.95-.96. Reliability estimates above .90 are the most desirable as they
indicate an assessment is highly reliable.

![](media/image4.wmf){width="4.6299212598425195in"
height="2.017322834645669in"}Figure 1 shows the growth in reading
comprehension scores by the random sample of first grade students in
Nebraska Reading First schools. The numbers represent the percentage of
first graders in the sample who scored at or above grade level on the
GORT comprehension subtest. In the spring of the first year of Reading
First implementation only 27% of these students were performing at or
above grade level on this measure. By the end of the third year of
Reading First implementation the percentage of first grade students
performing at or above grade level increased by 20%. Most of the
increase, however, was accomplished between year one and two, indicating
that need for further attention to comprehension.

A comparison of first grade performance in reading comprehension by
cluster membership is presented in figure 2. Cluster Two (low poverty,
low minority, smaller schools) has shown the largest and most consistent
growth since the first year of Reading First implementation (36%). This
clearly indicates that teachers in this cluster are
![](media/image5.wmf){width="4.008267716535433in"
height="2.4598425196850395in"}building on their instructional strengths
in this area year after year. Cluster Three (larger schools, high
poverty and ethnic minority student population) shows a fairly strong
growth trajectory with a 21% increase in percentage of first grade
students at or above grade level in comprehension. Given the very low
starting point (18% of students at or above grade level) and the high
mobility rate of students in this cluster this represents remarkable
instructional improvements by first grade teachers in cluster three.
Cluster One (high poverty, low minority) has shown the greatest
difficulty in sustaining the growth in performance of first grade
students on this measure after a gain of 12% in year two the numbers for
this cluster actually declined and resulted in only a 4% increase over
the three years. The results for this cluster explain in part the
limited overall growth between years two and three.

![](media/image6.wmf){width="3.5in" height="2.2688976377952756in"}

Achievement gaps in reading comprehension across the first three years
of Reading First implementation were analyzed between first grade
students who are classified as English language learners, minorities,
economically disadvantaged (FRL) and special education, and their peers
(see figures 3-6). The achievement gaps in first grade must be
interpreted with caution given the random sampling procedures related to
administration of this measure. The achievement gaps, however, do seem
consistent with growing discrepancies across categories. With the
exception of the slight narrowing of the gap between white and minority
students in the spring of 2007, the gaps are wider now than the first
year of Reading First implementation.

![](media/image7.wmf){width="3.6244094488188976in"
height="2.2783464566929132in"}The gaps for English language learners,
shown in figure 3, reveals a drop in percent of students at grade level.
This is a result of a uniquely successful ELL group of students that
were first grade students in spring 2005. Thus the 2006-2007 trend is a
better indicator of overall progress. This trend shows a small reduction
in the gap between English learners and their English only peers. Figure
4 shows that very little progress in reading comprehension was achieved
with students with disabilities. This result is indicative of a stronger
focus on basic phonological awareness and decoding skills with these
students. Most likely low decoding ability constrains their ability to
be successful in this assessment.

The gaps for ethnic minorities and economically disadvantaged students
in figures 5 and 6 show that both groups improved but that mainstream
students are currently benefitting more from Reading First instruction.
First grade results clearly highlight the need to focus more on
comprehension skills in first grade. The focus is even more crucial for
populations at-risk as their growth is not as strong as that of
mainstream students.

![](media/image8.wmf){width="3.499212598425197in"
height="2.25in"}![](media/image9.wmf){width="3.5in"
height="2.3751968503937007in"}

SECOND GRADE

Each spring all second and third grade students are administered the
Gates-MacGinitie comprehension assessment. Reliability estimates for
students in this age range are .91 for second grade and .93 for third
grade. This is indicative of a highly reliable assessment for students
at these grade levels.

Figure 7 shows the increase in the percentage of all second grade
students in Nebraska Reading First schools performing at or above grade
level in comprehension based on this assessment. Sixty-one percent of
these students were at or above grade level during the spring of 2005.
Growth between the first and second year of Reading First implementation
was stagnant with only a 1% increase in the percent of students
performing at or above grade level in reading comprehension. There was a
sizable increase between years two and three with a 13% increase.

![](media/image10.wmf){width="5.00984251968504in" height="2.3in"}

A comparison of second grade performance in reading comprehension was
also conducted by cluster membership (see figure 8). The growth
trajectories between all three clusters were relatively consistent
across all three years of implementation. There was very little change
in reading comprehension performance across all three clusters between
the first and second year of implementation. The most notable increase
in percentage of students at or above grade level on this measure
occurred between years two and three. Cluster Two had the largest
percentage increase (16%) compared to Cluster One and Cluster Three,
with 12% and 13% increases respectively. Cluster One and Cluster Two had
only 15% or less of second grade students performing below grade level
in reading ![](media/image11.wmf){width="4.45in"
height="2.4484251968503936in"}comprehension in the spring of 2007.

Achievement gaps in reading comprehension across the first three years
of Reading First implementation were analyzed between second grade
students who are classified as English language learners, minorities,
economically disadvantaged (FRL) and special education, and their peers.
The achievement gap between ELL and English only students narrowed
slightly during the second year of implementation, but was widened in
the spring of 2007 (see figure 9). As a result, the achievement gap
between these two groups of students has only decreased by 14% since
2005. The gap in reading comprehension between special education
students and their general education peers has increased since the first
year of Reading First implementation (see figure 10). The initial gap
between these two groups was 18% which has risen to nearly 30% in the
last school year.

![](media/image12.wmf){width="4.375196850393701in"
height="2.2216535433070868in"}![](media/image13.wmf){width="3.4098425196850393in"
height="2.20748031496063in"}

The gaps between white and minority students remained unchanged between
the first and second years of Reading First implementation but narrowed
by approximately 12% between years two and three (figure 11). The gap in
reading comprehension between economically disadvantaged (FRL) students
and their peers has actually increased by 5% since the first year of
implementation (see figure 12).

![](media/image14.wmf){width="4.400787401574803in"
height="2.293700787401575in"}![](media/image15.wmf){width="4.455511811023622in"
height="2.3283464566929135in"}

THIRD GRADE

![](media/image16.wmf){width="4.974409448818897in"
height="2.5811023622047244in"}Each spring all third grade students are
administered the Gates-MacGinitie comprehension assessment. Figure 13
shows the increase in percentage of third grade students performing at
or above grade level on this measure across the first three years of
Reading First implementation. As shown, the growth in reading
comprehension for this grade level has not improved tremendously since
the inception of Reading First. There was a 7% increase in the number of
third grade students performing at or above grade level between the
first and second year of implementation and only a 5% increase between
years two and three. Although there has been a 12% increase in the
number of third grade students performing at or above grade level in
reading achievement since the spring of 2005, it is clear that more
attention to this area of instruction is necessary in order to bolster
the reading comprehension skills of students at this grade level.

A comparison of third grade performance in reading comprehension was
also conducted by cluster membership (see figure 14). Third grade
students in Cluster Two have shown the greatest increase (16%) in the
percentage of students performing at or above grade level in reading
comprehension. There was an 11% increase in Cluster One and a 10% in
Cluster Three.

![](media/image17.wmf){width="4.5984251968503935in"
height="2.998031496062992in"}Although the difference between the
clusters is significant, the growth trajectories between Cluster One and
Cluster three are commensurate. The cluster analysis clearly shows the
need for greater instructional focus on reading comprehension strategies
in Cluster One and Cluster Three.

Achievement gaps in reading comprehension across the first three years
of Reading First implementation were analyzed between third grade
students who are classified as English language learners, minorities,
economically disadvantaged (FRL) and special education, and their peers
(see figure 16). Analysis of achievement gaps in reading comprehension
between ELL and English only students have narrowed by 17% since 2005.
The gap between special education students and their general education
peers has only decreased by 2 % since 2005 with a slight widening of the
gap in 2006.

![](media/image18.wmf){width="4.754724409448819in"
height="2.37007874015748in"}![](media/image19.wmf){width="3.72007874015748in"
height="2.4173228346456694in"}

The gap between minority and economically disadvantaged (FRL) students
and their peers have slightly widened since the inception of Reading
First in Nebraska, 4% and 5% respectively.

![](media/image20.wmf){width="4.6in"
height="2.3181102362204724in"}![](media/image21.wmf){width="4.628346456692913in"
height="2.3007874015748033in"}

SUMMARY

Reading comprehension scores in first through third grades have
increased fairly steadily since Reading First implementation began in
2005. Figure 19 shows the percentage of students performing at or above
grade level at each grade level. It is important to reiterate that the
measure and procedures used in first grade are different from those in
second and third grades. Since first grade comprehension assessments
involve random selection of a small number of students per grade level
the standard error of this assessment may be higher than in second and
third grades.

The percentage of students performing at or above grade level is the
highest for second grade (75%) followed by third grade at 64 percent.
Although first grade has the lowest percentage of students performing at
or above grade level in comprehension, they have shown the highest
percentage increase (20%) since the first year of Reading First
implementation.

![](media/image22.wmf){width="4.59251968503937in"
height="2.789763779527559in"}

Figure 20 shows the within cohort comparison in comprehension growth by
cluster. This figure represents the growth in student comprehension
performance of the cohort of students who were in first grade during the
first year of Reading First implementation. This figure shows the
greatest percentage increase in reading comprehension between first and
second grades with the growth leveling and even dropping off in third
grade. Differences in starting points and strength of growth between the
clusters are apparent. Students in Cluster Two have shown the most solid
and consistent growth in reading comprehension since 2005 with a 48%
increase in the percentage of students at or above grade level. This
level of performance is followed by Cluster One with a 40% increase.
Cluster Three has shown the least amount of growth with only a 24%
increase in the percent of students performing at or above grade level
in reading comprehension.

![](media/image23.wmf){width="4.275196850393701in"
height="3.341732283464567in"}

RECOMMENDATIONS

Although there have been increases in the percentage of students
performing at or above grade level in reading comprehension since the
inception of Reading First in Nebraska it is recommended that teachers
at all three grade levels increase the level of instructional focus
allocated to reading comprehension instruction. The aim of instructional
improvements in Nebraska Reading First schools should focus on reading
comprehension *throughout* the reading process. As stated by the
National Reading Panel (2000), "Comprehension is the reason for reading.
If readers can read the words but do not understand what they are
reading, they are not really reading" (p. 48). Effective comprehension
instruction supports the student by aiding the development of effective
strategies. These strategies include linking to prior knowledge before
reading, monitor comprehension during reading, and organize information
through the use of graphic organizers and semantic mapping throughout
the reading process. Comprehension is also supported by teaching
students how to recognize story structure, summarization, and question
generation.