# Wisconsin Reading First: 

# Year 4 (2006--07)

**November 2007**

# Submitted to:

**Nancy Anderson**

**Wisconsin Department of Public Instruction**

# Submitted by:

**Shazia Miller, Ph.D.**

**Andrew Swanlund**

**Jason Butler**

**Arianna Cisneros**

**Debra Davidson-Gibbs**

**Sarah Lessem**

![](media/image1.jpeg){width="2.872916666666667in"
height="0.6631944444444444in"}

> 20 North Wacker Drive, Suite 1231
>
> Chicago, IL 60606-2901
>
> 800-356-2735 312-288-7600
>
> www.learningpt.org
>
> 2213_11/07
>
> **Contents**

Page

Introduction 1

Program Description 1

Evaluation Overview 1

Evaluation Methods Overview 3

Summary of Evaluation Findings From Wisconsin Reading First Grant Years
2 and 3 4

Organization of the Report 6

Wisconsin Reading First Year 4 Program Implementation 7

> Program Implementation: Scientifically Based Reading Instruction 7
>
> Structures and Materials 8
>
> Perceptions of Instructional Change 22
>
> Use of Assessments 29
>
> Program Implementation: Professional Development 32
>
> Professional Development Provided by INSIGHT 33
>
> Professional Development Provided to the Reading First Coordinators 34
>
> Professional Development for Principals 35
>
> Teacher Participation in Professional Development 36
>
> Overall Reports of the Quality of Professional Development 40
>
> Program Implementation: Support for Successful Implementation 41
>
> District Support 42
>
> School-Level Support 42
>
> Monitoring of Literacy Instruction 45
>
> Sustainability of Reading First 46

Wisconsin Reading First Year 4 Program Effectiveness 48

> Progress Tracking Study 49
>
> Comparison Group Study 55
>
> Results 56
>
> Comparison of WKCE Performance in WRF and Similar Non-WRF Schools 57
>
> Impact of Level of Reported Implementation Factors on Student
> Achievement 58

Conclusions 61

> Structures and Materials 61
>
> Instructional Practice 61
>
> Use of Assessments 62
>
> Professional Development 62
>
> Monitoring and Support 62
>
> Reading First Schools Versus Statewide WKCE Performance 63
>
> Reading First Schools Versus Comparison Groups 63

References 64

Appendixes

Appendix A. Summary of Evaluation Activities: Grant Years 1 Through 4 A1

Appendix B. Sampling Procedure B1

Appendix C. Data Collection and Analysis C1

Appendix D. Wisconsin Elementary School Teacher Survey Frequency Tables
D1

Appendix E. Achievement Tables E1

Appendix F. Instruments F1

**Introduction**

**Program Description**

The federal Reading First program is a nationwide literacy initiative
specifically targeted at increasing the number of successful young
readers in Grades K--3. The program was established as part of the No
Child Left Behind (NCLB) Act with the goal of ensuring that all students
could read at or above grade level by the end of third grade. This goal
was to be achieved through the use of scientifically based reading
research (SBRR) methods and includes:

-   Use of SBRR methods and materials

-   Access to quality professional development

-   Use of assessments to monitor student performance and identify
    reading problems early

-   Use of methods and materials that include the five essential
    components of reading as identified by the National Reading Panel:
    phonemic awareness, phonics, fluency, vocabulary development, and
    comprehension.

Under the Reading First Program, the U.S. Department of Education
allotted funds to each state education agency (SEA) using a formula
based on the proportion of children between the ages of 5 and 17 living
in the state in families with incomes below the poverty line. The SEAs
then allotted the subgrants to local education agencies (LEAs) within
the state on a competitive basis. The SEAs were instructed to award the
most promising proposals based on SBRR with sufficient funds to allow
the LEAs to improve reading instruction. The states were required to
award each selected LEA with funds related to the number or percentage
of K--3 students in that LEA who were reading below grade level at the
time of the proposal.

In May 2003, the U.S. Department of Education approved the \$66 million,
six-year Wisconsin Reading First grant. The statewide impact goal of
Wisconsin Reading First is to ensure that every Wisconsin student can
read at or above grade level by the end of Grade 3, with a focus on
closing the achievement gaps between students of color, economically
disadvantaged students, and their peers. A total of 60 public schools
and two private schools receive an average of \$150,000 annually to
implement the Reading First program. The state provides a range of
Reading First professional development, which also is available to
non-Reading First schools. Wisconsin Reading First schools and school
districts use their direct funding to purchase instructional materials,
hardware, and software; to fund staff; and to provide professional
development.

**Evaluation Overview**

The evaluation of the Wisconsin Reading First program began in the
second year of the grant, which marked the first year of program
implementation. (Appendix A provides a detailed summary of evaluation
activities conducted during Years 1 through 4.) The evaluation focuses
on the implementation and effectiveness of Wisconsin Reading First. The
Wisconsin Reading First program evaluation consists of two major
components: program implementation and program effectiveness.

**Program Implementation**

The program implementation component evaluates the resources going into
Wisconsin Reading First--funded schools and monitors the instructional
processes that should be taking place in the classrooms as a result of
these resources. Data collected for this component of the evaluation
include classroom observations; interviews of teachers, principals, and
literacy staff; and a teacher survey at both treatment and control
schools to address relevant research questions.

Three broad research questions, with further subquestions, guide the
study of program implementation:

1.  To what extent is reading instruction in Wisconsin K--3 classrooms
    based on scientifically based reading research and the five
    essential components of reading? What instructional practices and
    materials do Reading First Schools use? Have instructional practices
    changed with program implementation? Do schools make significant use
    of student assessments and the data they can provide?

2.  To what extent has professional development influenced change in
    classroom reading instruction and the five essential components of
    reading? What professional development have Reading First schools
    received and is it of high quality, relevance, and utility?

3.  What factors have influenced successful Wisconsin Reading First
    implementation? What program and organizational support and
    monitoring do Reading First schools receive and to what extent will
    the program be sustainable after the grant ends?

**Program Effectiveness**

The program effectiveness component examines progress in student
achievement over time. Program effectiveness is measured both with
respect to changes in reading proficiency relative to state performance
standards and to the reading performance of students in similar schools
that do not receive Wisconsin Reading First funding. This component of
the evaluation uses student performance on standardized assessments and
the teacher survey to address the research questions.

The program effectiveness component embodies two distinct substudies:
The Progress Tracking Study examines the longitudinal achievement
obtained in general for Reading First schools, and the Comparison Group
Study specifically examines the growth within Reading First schools
relative to their comparison group peers.

**Progress Tracking Study.** The Progress Tracking Study is guided by
the following three evaluation questions:

-   What are the gains in reading proficiency for Wisconsin Reading
    First students disaggregated by gender, racial/ethnic minority
    status, limited-English-proficiency (LEP) status, disability status,
    economic status, and grade level?

-   How do gains in reading proficiency vary with the degree to which
    the essential components of reading have been implemented?

-   How do Wisconsin Reading First students compare with students
    statewide in terms of reading proficiency results on the Wisconsin
    Knowledge and Concepts Examination (WKCE; formerly the Wisconsin
    Reading Comprehension Test \[WRCT\]) at the end of Grade 3,
    disaggregated by gender, racial/ethnic minority status, LEP status,
    disability status, and economic status?

**Comparison Group Study.** Wisconsin Reading First effectiveness is
being studied using a longitudinal comparison group design. The
Comparison Group Study addresses the following evaluation questions:

-   Do Wisconsin Reading First students and schools demonstrate higher
    levels of reading achievement than demographically similar
    non-Wisconsin Reading First students and schools?

-   To what extent have Wisconsin Reading First schools been effective
    in reducing the number of students reading below the basic
    proficiency level compared to similar non-Wisconsin Reading First
    schools?

-   What factors contribute to differences in reading achievement
    between Wisconsin Reading First and non-Wisconsin Reading First
    schools?

**Evaluation Method Overview**

In spring 2007, Learning Point Associates conducted site visits to a
matched sample of Reading First and non-Reading First schools to explore
implementation of the program during Year 4. Site visits consisted of
observations of third grade literacy instruction and interviews with key
staff. School interviews at the 22 Reading First and non-Reading First
schools were guided by semistructured protocols developed using the
research questions and evaluation plan as a framework. Reading First
classroom teachers (*n =* 45 respondents), principals (*n =* 11), and
Reading First coordinators (*n =* 12) and non-Reading First classroom
teachers (*n =* 49), principals (*n =* 11), and literacy staff (*n =* 8)
were asked to comment on program. The Wisconsin Elementary School
Teacher Online Survey also was administered to a total of 879 Reading
First and 85 non-Reading First K--3 teachers from all K--3
English/language arts teachers in the 60 Reading First schools and all
K--3 English/language arts teachers in the 11 comparison schools. Each
method is described more fully below.

**Classroom observations** were conducted in 23 Reading First and 23
non-Reading First third grade classrooms using the Center for the
Investigation of Early Reading Achievement (CIERA) observation system to
provide coherent snapshots of literacy instruction along seven
dimensions: who provides instruction, instructional groupings, academic
areas addressed, specific literacy activities of the teacher,
instructional materials used, intervention style used, and expected
student response.

**Interviews** were conducted with key staff including Reading First
coordinators, principals, literacy staff, and teachers in 11 Reading
First and 11 non-Reading First schools. These interviews addressed class
size and teaching experience, changes in context, district support for
schools, program and organizational support and monitoring, program
overview, professional development, factors influencing implementation
of Reading First, plans to sustain the program model, and other
non-Reading First resources, programs, and initiatives. Table 1 exhibits
by staff position the distribution of interviews conducted in Year 4.

**Table 1. Spring 2007 Site Visit Interviews**

  ---------- ----------------------- ------------- --------------- -----------
  **Date**   **Staff Position**      **Reading     **Non-Reading   **Total**
                                     First**       First**         

  Spring     Teacher                 45            49              94
  2007                                                             

             Reading First           12            8               20
             coordinators/Literacy                                 
             staff                                                 

             Principals/Assistant    11            11              22
             Principals                                            

             Total                   68            68              136
  ---------- ----------------------- ------------- --------------- -----------

The Wisconsin Elementary School Teacher Online Survey was targeted at
all K--3 English/language arts teachers in the 60 Reading First schools
and all K--3 English/language arts teachers in the 11 comparison schools
selected for the study to assess the degree to which K--3 reading
curricula, reading instruction, and reading assessment practices are
being implemented as well as to confirm findings from staff interviews.

**Student achievement data for students across the state** were analyzed
to examine changes in students' achievement―both over time and compared
to students in non-Reading First schools.

Appendixes B and C provide further detail on data collection
instruments, analyses, and sampling procedures employed in the
evaluation.

**Summary of Evaluation Findings From Wisconsin Reading First\
Years 2 and 3**

The evaluation of Wisconsin Reading First extends over five years of the
six-year grant period. Grant Year 1 was used for planning and
pilot-testing the evaluation design and for preparing to evaluate
program implementation beginning in the 2004--05 school year (Grant Year
2). Beginning in 2004--05, the study of implementation and effectiveness
in a sample of Reading First schools began. Accordingly, Wisconsin
Reading First data collection began in Grant Year 2 and will continue
through Grant Year 5. The results from the evaluations of previous years
provide a context for understanding the Year 4 findings.

**Evaluation Year 2**

Year 2 of the Wisconsin Reading First grant was the first full year of
program implementation. Qualitative and quantitative data gathered
during Year 2 provided early information about the progress of Reading
First implementation in Wisconsin schools. Overall, teachers described
program professional development as necessary and useful for
incorporating the five essential components of reading into a structured
and effective instructional approach. The use of literacy centers was
described specifically by most teachers as productive for individual as
well as whole-group instruction. Teachers reported that Reading First
materials provided them with the opportunity to reach students by using
a more extensive repertoire of effective instructional approaches.

Program participants reflected on the early progress of the Reading
First implementation. The most commonly reported factor aiding in the
implementation of the program was the intense involvement of the
district and the state during the start-up phase of the program. This
upper level involvement helped to support schools as they implemented
the program model. The concern most often expressed by teachers and
principals about implementation in Grant Year 2 was in regard to the
literacy block (90 to 120 minutes), which they viewed as too long for
many students to stay focused on literacy instruction. Although elements
of the assessment system generated some concern from teachers who
disliked the amount of time needed for conducting assessments, most
teachers generally were positive about the test results they received
for individual students. In Year 2, the Wisconsin Department of Public
Instruction (WDPI) changed the statewide criterion referenced tests to
comply with NCLB, and the Wisconsin reading assessment was changed from
the WRCT to the WKCE.

The general consensus for the first year of implementation was that the
program was a positive force for reading instruction. Teachers reported
that they were engaged in active learning during professional
development. Teachers and literacy specialists generally regarded the
school principals as supportive and positive in most schools. Principals
themselves indicated that they were seeing positive changes in their
schools. Reading First coordinators and literacy staff were supportive
of teachers and were reported to be key in supporting literacy
instruction and assessment. Most school-level staff members regarded the
professional development given by the WDPI as productive and
instrumental for successful program implementation.

**Evaluation Year 3**

Information gathered regarding Year 3 of the grant indicated that
support for classroom teachers in improving the effectiveness of their
literacy instruction had increased substantially from the prior year.
Overall, teachers reported working more collaboratively with the Reading
First coordinator in their schools, and they reported experiencing
increased comfort with the Reading First program implementation in
general.

When teachers were interviewed in the spring 2006, they reported being
much more confident implementing the Reading First program and were more
focused on the five essential components of reading than they had been
in the previous autumn. Teachers said that they also were able to use
the assessment data more effectively to guide instructional practice,
which is one of the underlying goals of the Reading First program.
Teachers reported that they were able see improvements in their own
instructional practices as well as improvements in the reading
achievement of their students. Many of these teachers were using
formative assessments to monitor their students' progress throughout the
course of the year and thus better inform their instructional practices.
The impact of Reading First professional development during Year 3 was
reported as positive and influential on improving instructional
practices. School staff reported that the skills and knowledge gained
from the professional development sessions helped to support them in
implementing the Reading First program model.

Results from the analysis of student assessment data indicated that
students from the Reading First schools were benefiting from the
program. Across the multiple assessment systems, Reading First students
exhibited improved achievement while students at non-Reading First
non-Reading First schools, in general, remained static.

**Organization of the Report**

This report is organized into three sections. The first describes the
program implementation component of the evaluation design. The second
section presents findings from the program effectiveness component of
the evaluation. The final section draws conclusions from the Year 4
evaluation.

**Wisconsin Reading First Year 4 Program Implementation**

The program implementation component evaluates the resources going into
Wisconsin Reading First-funded schools and monitors the instructional
processes that should be taking place in the classrooms as a result of
these resources. Data collected during Year 4 for this component of the
evaluation include:

-   Classroom observations of 23 Reading First and 23 non-Reading First
    third grade classrooms

-   Interviews of Reading First coordinators (*n =* 12), classroom
    teachers (*n =* 45), and principals (*n =* 11) as well as
    non-Reading First literacy staff (*n =* 8), classroom teachers\
    (*n =* 49), and principals (*n =*11)

-   An online elementary teacher survey given to all K--3
    English/Language Arts teachers at all 60 reading First schools in
    the state and K--3 English/Language Arts teachers in the 11
    non-Reading First comparison schools chosen for the study

The findings from the evaluation of the program implementation component
of Wisconsin Reading First in Year 4 are presented as follows, organized
by research question:

**Program Implementation: Scientifically Based Reading Instruction.** To
what extent is reading instruction in Wisconsin K--3 classrooms based on
scientifically based reading research and the five essential components
of reading? What instructional practices and materials do Reading First
Schools use? Have instructional practices changed with program
implementation? Do schools make significant use of student assessments
and the data they can provide?

**Program Implementation: Professional Development.** To what extent has
professional development influenced change in classroom reading
instruction and the five essential components of reading? What
professional development have Reading First schools received and is it
of high quality, relevance, and utility?

**Program Implementation: Support for Successful Implementation.** What
factors have influenced successful Wisconsin Reading First
implementation? What program and organizational support and monitoring
do Reading First schools receive and to what extent will the program be
sustainable after the grant ends?

**Program Implementation: Scientifically Based Reading Instruction**

To what extent is reading instruction in Wisconsin K--3 classrooms based
on scientifically based reading research and the five essential
components of reading? What instructional practices and materials do
Reading First Schools use? Have instructional practices changed with
program implementation? Do schools make significant use of student
assessments and the data they can provide?

Scientifically based reading instruction is examined through an
exploration of Reading First instructional structures and materials,
teachers' perceptions of instructional change, and use of assessments.
The following are the key findings:

-   Most staff (89 percent) interviewed at Reading First schools
    reported that the program met the needs of all students, both high-
    and low-achievers, in contrast with the 54 percent of respondents
    from non-Reading First schools who said the same.

-   Teachers in both Reading First and non-Reading First schools
    reported relatively high levels of proficiency in instructing
    students in the five essential components of reading, and both
    regularly instruct their students in these areas.

-   Reading First teachers in third grade spend less time on the five
    essential components than teachers in kindergarten, first, or second
    grade, suggesting that third grade teachers may find the material
    too basic for their students.

-   Reading First teachers utilized small group arrangements more often
    than did non-Reading First teachers, and were more often observed
    coaching students and engaging in higher order questions of students
    than their non-Reading First counterparts.

-   Students in Reading First classrooms tended to be more actively
    engaged than students in non-Reading First classrooms.

-   In 2007, almost all observed text material in both Reading First and
    non-Reading First classrooms was narrative in nature.

-   Teachers report that they value Reading First for having increased
    their knowledge about effective reading instruction, increased their
    collaboration with each other, increased instructional consistency
    within schools, and provided them with coordinators who provide
    meaningful feedback.

-   Reading First and non-Reading First teachers use assessment data
    primarily to plan instruction, group students, and determine
    interventions.

**Structures and Materials**

One way Wisconsin Reading First hopes to improve students' reading
achievement is by changing teachers' instructional practices. To examine
these questions, surveys of Reading First teachers (*n =* 879) and
non-Reading First teachers (*n =* 85) were used to provide a broad
picture of teachers' views of the impact of the program on instruction.
In addition, interviews with Reading First classroom teachers (*n =*
45), principals (*n =* 11), and Reading First coordinators\
(*n =* 12) provide details regarding the instructional materials
provided through Reading First, teachers' comfort levels with these
materials, the materials' alignment with program goals, and the use of
supplementary materials. Interviews with non-Reading First classroom
teachers\
(*n =* 49), principals (*n =* 11), and literacy staff (*n =* 8) provide
evidence of the counterfactual: are the reports of Reading First
participants unique to those involved with the program or common across
Wisconsin teachers? Finally, classroom observations of literacy
instruction provide direct evidence of the third-grade instructional
practices of Reading First teachers and how they compare to non-Reading
First teachers. The triangulation of all these pieces of information
suggests that Reading First is meeting the instructional needs of
teachers and the educational needs of students. This section provides
details pertaining to coordination with instructional goals,
implementation of the literacy block, coverage of the five essential
components of reading, descriptions of the core reading program's
specifics regarding instructional practice, and descriptive analyses of
literacy instruction by content and structure.

**Coordination With Instructional Goals.** The Reading First initiative
encompasses several major instructional goals, including the use of
comprehensive reading instruction based on SBRR methods and an emphasis
on the five essential components of early reading instruction.

Reading First teachers were asked in the survey to rate the extent to
which the Reading First initiative goals align with their instructional
goals. Overall, a large majority (86 percent) of teachers at Reading
First schools reported that the goals of the program aligned with their
instructional goals to either a moderate or substantial degree. Table 2
presents the extent to which Reading First goals align with individual
teachers' instructional goals.

Table 2. Extent to Which Reading First Goals Align With Teacher
Instructional Goals

+----------------------------------------------------------+-----------+
| Reading First Schools                                    | Frequency |
|                                                          |           |
|                                                          | *n* = 804 |
+----------------------------------------------------------+-----------+
| Not applicable (assessment team used in our school)      | 1.9%      |
+----------------------------------------------------------+-----------+
| Applicable, but do not align with my instructional goals | 1.2%      |
| at all                                                   |           |
+----------------------------------------------------------+-----------+
| Align with my instructional goals to some degree         | 11.2%     |
+----------------------------------------------------------+-----------+
| Align with my instructional goals to a moderate degree   | 39.9%     |
+----------------------------------------------------------+-----------+
| Align with my instructional goals substantially          | 45.8%     |
+----------------------------------------------------------+-----------+

Among the specific instructional goals laid out in the Reading First
proposal is to meet the needs of all students, regardless of their
ability levels. Therefore, interviewers asked principals, Reading First
coordinators, and teachers whether their literacy program meets the
needs of both low- and high-achieving students. Most respondents (89
percent) agreed that their literacy program does meet the needs of high-
and low-achievers alike, citing the accelerated readers, interventions,
and pull-out programs as ways to differentiate instruction. One
respondent noted that, "It's really helped us diversify our instruction
and make sure that we are targeting all three tiers." A smaller
percentage (54 percent) of the respondents interviewed at non-Reading
First schools said the curriculum meets both high- and low-achieving
students' needs.

However, a minority (12 percent) of Reading First staff, particularly
principals (36 percent), said they do not believe the program meets the
needs of high-achieving students. Teachers often noted that although
there are resources available through Reading First that would meet the
needs of high-achieving students, it took more initiative on the part of
teachers to incorporate them into classroom instruction and that this
was a continual challenge.

**Implementation of the Literacy Block.** The literacy block is
uninterrupted instructional time dedicated to literacy instruction. The
length of literacy sessions among schools varied slightly, with all
teachers interviewed in Reading First schools reporting that their
literacy sessions were at least 61--90 minutes or longer, while 14
percent of non-Reading First schools had literacy blocks of less than an
hour. Nearly half (45 percent) reported that their literacy sessions
were two hours or longer, as compared to 34 percent of teachers in
non-Reading First schools.

Of the Reading First teachers interviewed, 39 percent described having
literacy sessions that were broken up into morning and afternoon blocks,
with a 90-minute morning session and a\
30-minute afternoon session being reported most often (18 percent). Ten
percent of non-Reading First teachers similarly reported using morning
and afternoon blocks. Teachers who use morning and afternoon blocks
often said that it enabled them to incorporate supplemental reading
enrichment. Some teachers said that they use the morning for
uninterrupted direct instruction; they then use reading centers, writing
activities, and interventions in the afternoon. Table 3 exhibits
self-reports by teachers from Reading First and non-Reading First
teachers in regard to the length of their literacy block.

Table 3. Length of Literacy Block

  --------------------------- --------------------- ----------------------
  **Length of Time (in        **Reading First (*n*  **Non-Reading First
  Minutes)**                  = 44)**               (*n* = 49)**

  Less than 45                0.0%                  2.0%

  46--60                      0.0%                  12.2%

  61--90                      13.6%                 16.3%

  91--120                     18.2%                 32.7%

  More than 120               29.5%                 24.5%

  Morning and afternoon       15.9%                 10.2%
  (\>120)                                           

  Morning and afternoon       22.7%                 0.0%
  (90--120)                                         
  --------------------------- --------------------- ----------------------

In terms of total time spent on reading instruction, most respondents to
the teachers' survey from both Reading First and non-Reading First
schools reported spending 10 or more hours per week teaching reading or
language arts to their classes (74 percent and 67 percent,
respectively). Teachers from Reading First schools were more likely to
report spending more than 12 hours per week on literacy instruction than
were those from non-Reading First schools (38 percent compared to 24
percent).

Teachers from both Reading First and non-Reading First schools reported
no change since the 2005-2007 school year on the amount of time they
spend teaching literacy, saying on the survey that they spend the same
amount of time teaching literacy this year as they had last year (55
percent and 75 percent, respectively). However, 44 percent of
respondents at Reading First schools and 22 percent of those at
non-Reading First schools indicated that the amount of time they spent
teaching has increased since last year.

Respondents to the teachers' survey were asked what percentage of the
120-minute reading block (90 minutes uninterrupted reading and 30
minutes intervention) was spent doing a variety of activities (see
Tables D26a and D26b in Appendix D). Of the 10 choices, teachers from
Reading First schools reported spending the highest mean percentages of
time providing direct instruction, working with students in guided
reading or writing, and engaging students in the reading process (16
percent, 15 percent, and 13 percent, respectively). Teachers from
non-Reading First schools reported similar patterns of practice,
spending providing direct instruction (14 percent), working with
students in guided reading or writing (16 percent), and engaging
students in the reading process (13 percent).

Teachers from Reading First schools and non-Reading First schools
indicated on the survey that they spend a higher mean percentage of
their literacy block working on reading skills and strategies (57
percent and 53 percent, respectively) than on appreciation and analysis
of literature and student writing. Non-Reading First teachers spent a
greater percentage of the reading block on writing than did Reading
First teachers. Table 4 exhibits the percentage of time spent by
teachers at Reading First and non-Reading First schools.

Table 4. Percentage of Time Spent on Specific Activities During the
Reading Block

+---------------------------------+-----------------+------------------+
| Primary Focus                   | **Reading First | Non-Reading      |
|                                 | Teachers**      | First Teachers   |
|                                 |                 |                  |
|                                 | ***n* = 873**   | *n* = 85         |
+---------------------------------+-----------------+------------------+
| Reading skills and strategies   | 56.8%           | 52.5%            |
+---------------------------------+-----------------+------------------+
| Appreciation and analysis of    | 20.1%           | 21.7%            |
| literature                      |                 |                  |
+---------------------------------+-----------------+------------------+
| Student writing                 | 14.5%           | 23.5%            |
+---------------------------------+-----------------+------------------+

**Coverage of the Five Essential Components of Reading.** The National
Reading Panel distilled years of literacy research into five essential
components of early reading instruction: phonemic awareness, phonics,
fluency, vocabulary development, and comprehension. The Reading First
program model includes extensive focus on these components to improve
student reading achievement. In all, teachers are utilizing the five
essential components to improve their literacy instruction.

When asked in the online survey to indicate how often their students
received instruction in the five essential components of reading, the
majority of the teachers from Reading First schools indicated that they
cover each component daily (see Table D24a in Appendix D). Both Reading
First and non-Reading First teachers were most likely to report that
their students receive instruction in comprehension daily (76 percent
and 72 percent, respectively) and were least likely to report that their
students receive instruction in phonemic awareness daily (59 percent and
51 percent, respectively).

Looking across grades, an analysis of variance (ANOVA) of trends from
the 2006 to the 2007 online teachers' surveys shows that Grade 3
teachers reported significantly less emphasis on the five essential
components than did Kindergarten, Grade 1, and Grade 2 teachers (*F* =
5.2, p \< 0.01) (see Figure 1). This difference is evident in Figure 1
across both years as the line for\
Grade 3 teachers is considerably lower than the lines for the other
three grades.

**Figure 1. Reported Use of Five Essential Components\
During the 120-Minute Literacy Block by Grade Level**

![](media/image2.wmf){width="6.010416666666667in"
height="4.197916666666667in"}

Between 57 percent and 84 percent of the Reading First teachers who took
the teacher survey reported giving moderate to sustained emphasis to
each of the five essential components of reading (see Table 5). Half the
survey respondents at Reading First schools reported that there was a
sustained emphasis on comprehension during their reading block, whereas
only 29 percent of these individuals reported a sustained emphasis on
phonemic awareness.

Table 5. Emphasis During the 120-Minute Reading Block\
on Each of the Five Essential Components of Reading

  ----------------------- ----------- ------------ ------------ -------------
  Reading First Schools   **No        **Slight     **Moderate   Sustained
                          Emphasis\   Emphasis\    Emphasis\    Emphasis\
                          (None of\   (\<25% of\   (25--35% of\ (\>35% of\
                          the Time)** the Time)**  the Time)**  the Time)

  Phonemic awareness (*n* 6.6%        36.9%        27.5%        29.1%
  = 819)                                                        

  Phonics (*n* = 821)     3.0%        24.5%        35.0%        37.5%

  Fluency (*n* = 823)     2.2%        19.9%        39.4%        38.5%

  Vocabulary (*n* = 821)  1.7%        23.5%        40.0%        34.8%

  Comprehension (*n* =    1.9%        13.9%        34.2%        50.0%
  822)                                                          
  ----------------------- ----------- ------------ ------------ -------------

The non-Reading First teachers who participated in the survey were more
likely to report placing a moderate or sustained emphasis on
comprehension, vocabulary, and phonemic awareness than Reading First
teachers (see Table 6). Within non-Reading First schools, 58 percent of
teachers who took the online survey indicated they place a sustained
emphasis on comprehension. At least 61 percent of teachers reported that
they place either a moderate or sustained emphasis on each of the five
essential components of reading, with almost all (94 percent) reporting
this for the focus on comprehension.

Table 6. Emphasis During Reading Instruction Spent on Each of\
the Five Essential Components of Reading

+--------------------------+---------+-----------+----------+----------+
| Non-Reading First        | **No    | **Slight  | **       | **S      |
| Schools                  | Emp     | E         | Moderate | ustained |
|                          | hasis** | mphasis** | Em       | Em       |
|                          |         |           | phasis** | phasis** |
|                          | **(None | **(\<25%  |          |          |
|                          | of\     | of\       | **       | (\>35%   |
|                          | the     | the       | (25--35% | of\      |
|                          | Time)** | Time)**   | of\      | the      |
|                          |         |           | the      | Time)    |
|                          |         |           | Time)**  |          |
+--------------------------+---------+-----------+----------+----------+
| Phonemic awareness *(n*  | 3.6%    | 35.7%     | 29.8%    | 31.0%    |
| = 84)                    |         |           |          |          |
+--------------------------+---------+-----------+----------+----------+
| Phonics (*n* = 84)       | 2.4%    | 31.0%     | 27.4%    | 39.3%    |
+--------------------------+---------+-----------+----------+----------+
| Fluency (*n* = 84)       | 3.6%    | 28.6%     | 39.3%    | 28.6%    |
+--------------------------+---------+-----------+----------+----------+
| Vocabulary (*n* = 84)    | 2.4%    | 17.9%     | 39.3%    | 40.5%    |
+--------------------------+---------+-----------+----------+----------+
| Comprehension (*n* = 84) | 2.4%    | 3.6%      | 35.7%    | 58.3%    |
+--------------------------+---------+-----------+----------+----------+

When asked in the online survey to rate how proficient they consider
themselves to be at teaching each of the five essential components of
reading, most Reading First and non-Reading First teachers reported that
they felt moderately or extremely proficient at instructing students in
each of the five essential components (see Table 7). Larger percentages
of the Reading First teachers reported being moderately or extremely
proficient in phonics, vocabulary development, and comprehension (87
percent each) than in phonemic awareness or fluency (80 percent and 84
percent, respectively). Similarly, more than 90 percent of the
non-Reading First staff considered themselves to be moderately or
extremely proficient in phonics, vocabulary, and comprehension, and
phonics (92 percent, 94 percent, and 94 percent, respectively). More
than three quarters said they were either moderately or extremely
proficient at teaching phonemic awareness (86 percent) and fluency (85
percent) (see Tables 7 and 8).

Table 7. Reading First: Reported Proficiency in Teaching\
the Five Essential Components of Reading

  ------------------- -------------- --------------- -------------- -------------- --------------
  Reading First       **Not          **Applicable,   **Somewhat     **Moderately   **Extremely
  Schools             Applicable**   but Not         Proficient**   Proficient**   Proficient**
                                     Proficient at                                 
                                     All**                                         

  Phonemic awareness\ 1.7%           1.2%            16.6%          47.4%          33.1%
  (*n* = 827)                                                                      

  Phonics (*n* = 827) 0.7%           0.7%            11.4%          46.9%          40.3%

  Fluency (*n* = 827) 0.7%           0.6%            14.6%          52.0%          32.0%

  Vocabulary (*n* =   0.5%           0.7%            12.0%          53.8%          33.0%
  825)                                                                             

  Comprehension (*n*  0.5%           0.7%            10.5%          50.5%          37.7%
  = 825)                                                                           
  ------------------- -------------- --------------- -------------- -------------- --------------

Table 8. Non-Reading First: Reported Proficiency in Teaching\
the Five Essential Components of Reading

  ----------------- -------------- --------------- -------------- -------------- --------------
  Non-Reading First **Not          **Applicable,   **Somewhat     **Moderately   **Extremely
  Schools           Applicable**   but Not         Proficient**   Proficient**   Proficient**
                                   Proficient at                                 
                                   All**                                         

  Phonemic          0.0%           1.2%            13.1%          50.0%          35.7%
  awareness\                                                                     
  (*n* = 827)                                                                    

  Phonics (*n* =    0.0%           0.0%            8.3%           47.6%          44.0%
  827)                                                                           

  Fluency (*n* =    0.0%           2.4%            13.1%          42.9%          41.7%
  827)                                                                           

  Vocabulary (*n* = 0.0%           0.0%            6.0%           48.8%          45.2%
  825)                                                                           

  Comprehension\    0.0%           1.2%            4.8%           44.6%          49.4%
  (*n* = 825)                                                                    
  ----------------- -------------- --------------- -------------- -------------- --------------

As Figure 2 shows, teachers in both survey groups increased in their
reported proficiency at teaching the five essential components. Similar
to the first administration of the survey, control teachers reported
more proficiency at teaching the components (*F* = 7.1, p \< 0.01).

**Figure 2. Reported Proficiency at Teaching the Five Essential
Components of Reading**

![](media/image3.wmf){width="6.010416666666667in"
height="4.197916666666667in"}

## Core Reading Program. Reading First and non-Reading First schools use a variety of core reading programs and supplemental materials to cover literacy instruction. 

For the most part, respondents at both Reading First and non-Reading
First schools appeared to be satisfied with their core reading program.
Teachers and literacy staff from both Reading First schools and
non-Reading First schools reported similar strengths for the respective
programs used by their schools. One common factor cited by all
respondents is that the programs they used provide a good mix of
content, materials, and activities. Respondents at both Reading First
and non-Reading First schools also frequently mentioned that they liked
that the program covered one or several of the reading components well
and that it allowed them to differentiate instruction.

When asked what aspects of reading they did not think were adequately
covered by the program, the responses were so varied that there were no
clear-cut or frequently cited answers among or between Reading First and
non-Reading First staff. Some teachers expressed a desire for more
high-interest reading materials that would engage their students, while
others said the materials were too challenging. Similarly, some teachers
said that they felt limited by the program, while others responded that
the program was overwhelming and that they had a difficult time figuring
out what activities and materials to use.

In the Reading First schools, the core reading program that was most
commonly used was provided by Houghton Mifflin. Direct instruction also
was cited frequently, as was Harcourt's *Trophies* series. All the
schools appear to have been using the same program for three years or
longer, presumably at least since the inception of Reading First in
2003. In contrast, the core reading programs used in non-Reading First
schools were much more varied. Harcourt programs, including *Trophies*,
were the most frequently cited. Other programs that were commonly used
include direct instruction, Houghton Mifflin, Success for All, and
Macmillan/McGraw-Hill, among others. Most non-Reading First schools also
appear to have been using the programs for three years or longer,
although at least two of the schools said they had gotten a new program
in the previous two years.

Principals, Reading First coordinators, and teachers at Reading First
schools overwhelmingly reported that core reading program materials
fully meet the needs of their students (91 percent, 75 percent, and 84
percent, respectively). Furthermore, with the exception of two teachers,
the remaining staff commented that the Reading First program meets their
students' needs to at least some degree. One teacher from a Reading
First school explained his/her satisfaction with available core
materials:

> I guess I feel these kids are very lucky, because they have so much
> available to them right now. Without the Reading First grant, I'd
> really feel us as teachers wouldn't have been as effective. The
> training that we've received, and the ideas we've gotten, the
> activities, and things received through money from the grant have
> greatly broadened us as teachers. And the students, I think, are
> really lucky.

The staff of non-Reading First schools also were quite satisfied with
their core program, although somewhat less so than the staff at Reading
First schools. When asked if the reading program met their needs, the
majority (71 percent) of non-Reading First teachers said that it did.
Slightly more than half the principals (55 percent) said that it did
meet their teachers' needs, 27 percent said that it did somewhat, and
the remaining 18 percent responded that it did not meet teacher needs.
No literacy staff responded that the program did not meet teacher needs,
38 percent thought that it somewhat met their needs, and 62 percent
thought that it fully met their needs.

More than two thirds of Reading First teachers (68 percent) and
principals (67 percent) felt that the reading program fully meets
teachers' needs, while 75 percent of Reading First coordinators believed
so as well. Despite the general satisfaction with core reading
materials, many reported feeling that they were limited in their
discretion as to what materials to use. When asked how much discretion
they had in following the core reading program, at least half the
teachers (55 percent) and Wisconsin Reading First coordinators (50
percent) responded that they had some discretion. Although 21 percent of
teachers and 25 percent of Reading First coordinators said that they had
no or limited discretion in how they used the core reading materials, an
equal number of teachers and Reading First coordinators (21 percent and
25 percent, respectively) reported having a lot of discretion and the
freedom to supplement the materials as they saw fit.

Similarly, at non-Reading First schools, nearly half the teachers (49
percent) and literacy staff (50 percent) said that teachers were given
some discretion in following the core reading program. Ten percent of
teachers said they had total discretion, while only one literacy staff
person agreed. Another 33 percent of teachers said they had a lot of
discretion, while 25 percent of the literacy staff reported a lot of
teacher discretion as well. Only 8 percent of teachers said they had no
discretion in following the program, and one literacy staff member
agreed. Figure 3 exhibits discretion by treatment group.

**Figure 3. Core Reading Program Teacher Discretion**

![](media/image4.png){width="5.885416666666667in" height="3.59375in"}

Although several teachers indicated they had little discretion over the
program, more than half of the teachers at Wisconsin Reading First
schools (61 percent) reported that they made changes to the program,
usually in the form of adding materials. Only 31 percent of non-Reading
First teachers said the same. They said these changes were made to
better meet student needs, or at times because they had increased funds
or wanted to update the versions they used. They reported that the
overall effects were that students responded well to the materials and
that their teaching improved.

When asked to specify what other materials they use to teach reading,
Reading First teachers and Reading First coordinators most frequently
reported the use of leveled readers (50 percent), trade books (33
percent), books from the classroom libraries (18 percent), and
technology such as library centers and software programs (24 percent).
To assist struggling readers, Reading First teachers and Reading First
coordinators most often cited the use of pullout techniques\
(62 percent), additional materials such as guided readers (38 percent),
and Title 1 services and resources (25 percent). They also cited the
assistance of other staff, aides, and volunteers\
(15 percent) and the use of technology and software programs (11
percent).

Non-Reading First teachers had largely similar responses. In interviews,
Non-Reading First teachers and literacy staff also frequently cited the
use of leveled readers (47 percent), trade books (46 percent), some sort
of supplemental books (44 percent), and technology (19 percent). Similar
to their counterparts in Reading First schools, they frequently stated
that they provide struggling readers with Title 1 services and resources
(48 percent), additional materials such as guided readers (41 percent),
use of pullout techniques (13 percent), and the assistance of other
staff, aides, and volunteers (32 percent). They stated that they also
use technology and software programs (16 percent) to help struggling
readers.

The survey asked teachers to report how often they had used a variety of
materials in their reading program in the past school year. There were
no materials that a majority of the Reading First teachers reported
using daily. However, teachers at Reading First schools were most likely
to report the daily use of student writing (41 percent), narrative
textbooks (37 percent), manipulatives (35 percent), and teacher-made
materials (35 percent). A large majority of teachers from Reading First
schools (81 percent) indicated that they consider themselves moderately
or extremely proficient at using the Reading First instructional
materials. (For a full breakdown of the materials used, see Tables 29a
and 29b in Appendix D.)

Many (44 percent) of the survey respondents from non-Reading First
schools indicated that they used student writing during 2006-07 on a
daily basis. Generally speaking, teachers at non-Reading First schools
were most likely to report using student writing (82 percent),
teacher-made materials (71 percent), children's narrative textbooks (66
percent), narratives (65 percent), and worksheets (65 percent).

**Instructional Practice.** Each year of the Reading First evaluation
classroom observations was conducted to systematically collect
information on classroom instruction. Kindergarten and first grade
classrooms were observed in spring 2005; second grade and third grade
observations occurred in spring 2006 and spring 2007, respectively. This
systematic observation of different grades allows for the development of
a profile of instruction a student might receive during the K--3 years
in a Reading First school. It also allows for comparison between two
closely related grades. As such, after the presentation of the
four-grade profile of the five essential components, results are
presented for Reading First and non-Reading First schools over the last
two years, during which second grade and third grade were observed.

Observations of K--3 classrooms from 2005 to 2007 indicate a change in
the focus of instruction as students progress through the four grades.
As shown in Table 9, phonemic awareness appears to be stressed in
Kindergarten (17 percent of instructional focus minutes), but then did
not constitute a significant part of instruction for Grades 1, 2, and 3.
In addition, focus on phonics and word recognition decreased over the
years, whereas comprehension, vocabulary development, and fluency were
observed more often in the later grades.

Table 9. Content Focus (Percentage of Minutes Observed) by Grade

+----------------------------+---------+---------+---------+---------+
| **Content Focus**          | **K**   | **1**   | **2**   | **3**   |
|                            |         |         |         |         |
|                            | **Min = | **Min = | **Min = | **Min = |
|                            | 227**   | 263**   | 889**   | 579.5** |
+----------------------------+---------+---------+---------+---------+
| Phonemic awareness         | 17.2%   | 0.0%    | 0.3%    | 0.5%    |
+----------------------------+---------+---------+---------+---------+
| Phonics                    | 25.6%   | 23.6%   | 1.6%    | 14.0%   |
+----------------------------+---------+---------+---------+---------+
| Word recognition           | 16.3%   | 15.2%   | 0.0%    | 6.4%    |
+----------------------------+---------+---------+---------+---------+
| Comprehension              | 8.4%    | 21.3%   | 46.2%   | 31.2%   |
+----------------------------+---------+---------+---------+---------+
| Vocabulary                 | 5.7%    | 3.0%    | 18.4%   | 17.2%   |
+----------------------------+---------+---------+---------+---------+
| Reading connected text     | 23.3%   | 35.7%   | 27.4%   | 18.3%   |
| (not turn-taking)          |         |         |         |         |
+----------------------------+---------+---------+---------+---------+
| Fluency                    | 3.5%    | 1.1%    | 6.0%    | 12.4%   |
+----------------------------+---------+---------+---------+---------+

Observations conducted in 2006 and 2007 of second- and third-grade
classrooms documented teachers' use of a variety of instructional
approaches, including various types of grouping (see Table 10). Teachers
group students in many ways to better differentiate instruction and to
heighten student engagement. Reading First Schools utilized small group
arrangements more often than did non-Reading First group schools. In
addition, differences between Reading First and non-Reading First
schools increased from 2006 to 2007. That is, in 2006, roughly 57
percent of Reading First observational segments had a large group
format, compared to 69 percent of non-Reading First school segments. In
2007, this percentage declined slightly for Reading First schools but
increased for non-Reading First schools; 54 percent of Reading First
segments had a large group format compared to 73 percent of non-Reading
First school segments in 2007.

Table 10. Student Grouping (Percentage of Observational Segments)\
by Grade and Treatment Group

+----------------------+-----------+-----------+-----------+-----------+
| **Grouping**         | **2006    |           | **2007    |           |
|                      | Second    |           | Third     |           |
|                      | Grade**   |           | Grade**   |           |
+----------------------+-----------+-----------+-----------+-----------+
|                      | **RF**    | **Comp**  | **RF**    | **Comp**  |
|                      |           |           |           |           |
|                      | ***N* =   | ***N* =   | ***N* =   | ***N* =   |
|                      | 264**     | 281**     | 180**     | 179**     |
+----------------------+-----------+-----------+-----------+-----------+
| Large group          | 57.2%     | 68.7%     | 53.9%     | 73.2%     |
+----------------------+-----------+-----------+-----------+-----------+
| Small group          | 30.7%     | 18.1%     | 27.8%     | 11.2%     |
+----------------------+-----------+-----------+-----------+-----------+
| Pairs                | 6.4%      | 4.3%      | 6.1%      | 3.9%      |
+----------------------+-----------+-----------+-----------+-----------+
| Individual           | 5.7%      | 7.1%      | 7.2%      | 11.7%     |
+----------------------+-----------+-----------+-----------+-----------+
| None                 | 0.0%      | 1.8%      | 5.0%      | 0.0%      |
+----------------------+-----------+-----------+-----------+-----------+

[]{.mark}

For both Reading First and non-Reading First schools, teachers were more
likely to use narrative rather than informational texts, as shown in
Table 11. For example, in 2007 almost all observed third-grade teachers
(96 percent in Reading First Schools, and 93 percent in non-Reading
First schools) were using narrative text materials during the periods
they were observed. From 2006 to 2007 the use of narrative texts
increased for both groups.

Table 11. Text Type Observed (Percentage of Minutes Observed)\
by Grade and Treatment Group

+----------------------+-----------+-----------+-----------+-----------+
| **Text Type**        | **2006    |           | **2007    |           |
|                      | Second    |           | Third     |           |
|                      | Grade**   |           | Grade**   |           |
+----------------------+-----------+-----------+-----------+-----------+
|                      | **RF**    | **Comp**  | **RF**    | **Comp**  |
|                      |           |           |           |           |
|                      | **Min =   | **Min =   | **Min =   | **Min =   |
|                      | 633.5**   | 799**     | 360**     | 404**     |
+----------------------+-----------+-----------+-----------+-----------+
| Narrative text       | 79.3%     | 89.4%     | 95.8%     | 92.6%     |
+----------------------+-----------+-----------+-----------+-----------+
| Informational text   | 20.7%     | 10.6%     | 4.2%      | 7.4%      |
+----------------------+-----------+-----------+-----------+-----------+

[]{.mark}

Teachers used a variety of strategies to interact with their students
during the literacy block. In 2007, similar levels of recitation and
modeling were observed in the Reading First and non-Reading First
classrooms, as shown in Table 12. Reading First teachers were more often
observed coaching students and engaging in higher order questioning of
students. However, non-Reading First teachers were more often observed
listening and watching their students; non-Reading First teachers also
engaged in fewer "telling" activities, or activities where the teacher
explains how to do something, than were Reading First teachers.

Table 12. Teacher Interaction Activity (Percentage of Minutes Observed)\
by Grade and Treatment Group

+----------------------+-----------+-----------+-----------+-----------+
| **Teacher            | **2006    |           | **2007    |           |
| Interaction**        | Second    |           | Third     |           |
|                      | Grade**   |           | Grade**   |           |
+----------------------+-----------+-----------+-----------+-----------+
|                      | **RF**    | **Comp**  | **RF**    | **Comp**  |
|                      |           |           |           |           |
|                      | **Min =   | **Min =   | **Min =   | **Min =   |
|                      | 1,367.5** | 1,335.5** | 1,154.5** | 1,102**   |
+----------------------+-----------+-----------+-----------+-----------+
| Telling              | 28.7%     | 17.6%     | 25.2%     | 19.5%     |
+----------------------+-----------+-----------+-----------+-----------+
| Recitation           | 10.9%     | 14.8%     | 5.7%      | 5.3%      |
+----------------------+-----------+-----------+-----------+-----------+
| Modeling             | 3.7%      | 3.2%      | 4.4%      | 4.4%      |
+----------------------+-----------+-----------+-----------+-----------+
| Coaching             | 4.1%      | 7.0%      | 21.4%     | 18.4%     |
+----------------------+-----------+-----------+-----------+-----------+
| Higher order         | 10.7%     | 10.4%     | 15.5%     | 10.0%     |
| questioning          |           |           |           |           |
+----------------------+-----------+-----------+-----------+-----------+
| Lower order          | 20.9%     | 23.8%     | 17.4%     | 26.7%     |
| questioning          |           |           |           |           |
+----------------------+-----------+-----------+-----------+-----------+
| Listening/watching   | 21.1%     | 23.2%     | 10.4%     | 15.7%     |
+----------------------+-----------+-----------+-----------+-----------+

[]{.mark}

Reading First third-grade students tended to be more actively engaged in
2007 than were non-Reading First third graders (43 percent of
observational segments in Reading First classrooms compared with 26
percent in non-Reading First group classrooms). This marks a
considerable change over 2006 where an identical percentage of active
(38 percent) and passive (63 percent) student activity was observed for
both groups of second-grade classrooms. In particular, students in
Reading First classrooms tended to spend more time writing and reading
aloud together than did their counterparts in non-Reading First group
classrooms (see Table 13 and Figure 4).

Table 13. Student Activity (Percentage of Minutes Observed)\
by Grade and Treatment Group

+----------------------+-----------+-----------+-----------+-----------+
| **Student Activity** | **2006    |           | **2007    |           |
|                      | Second    |           | Third     |           |
|                      | Grade**   |           | Grade**   |           |
+----------------------+-----------+-----------+-----------+-----------+
|                      | **RF**    | **Comp**  | **RF**    | **Comp**  |
|                      |           |           |           |           |
|                      | **Min =   | **Min =   | **Min =   | **Min =   |
|                      | 1,505.5** | 1,722**   | 961.5**   | 964**     |
+----------------------+-----------+-----------+-----------+-----------+
| **Total Active**     | **37.5%** | **37.5%** | **43.2%** | **26.0%** |
+----------------------+-----------+-----------+-----------+-----------+
| Reading aloud        | 9.2%      | 7.2%      | 11.1%     | 4.4%      |
| together             |           |           |           |           |
+----------------------+-----------+-----------+-----------+-----------+
| Responding together  | 7.9%      | 2.9%      | 8.6%      | 9.8%      |
+----------------------+-----------+-----------+-----------+-----------+
| Writing              | 17.5%     | 24.2%     | 21.1%     | 11.5%     |
+----------------------+-----------+-----------+-----------+-----------+
| Manipulating         | 2.9%      | 3.2%      | 2.4%      | 0.3%      |
+----------------------+-----------+-----------+-----------+-----------+
| **Total Passive**    | **62.5%** | **62.5%** | **56.8%** | **74.0%** |
+----------------------+-----------+-----------+-----------+-----------+
| Reading turn-taking  | 15.8%     | 8.9%      | 17.4%     | 17.4%     |
+----------------------+-----------+-----------+-----------+-----------+
| Turn-taking          | 29.0%     | 32.4%     | 23.9%     | 35.9%     |
+----------------------+-----------+-----------+-----------+-----------+
| Listening            | 17.7%     | 21.2%     | 15.5%     | 20.7%     |
+----------------------+-----------+-----------+-----------+-----------+

[]{.mark}

**Figure 4. Active Versus Passive Learning (Percentage of Minutes
Observed)\
by Grade and Treatment Group**

![](media/image5.wmf){width="5.999305555555556in"
height="3.5555555555555554in"}

**Teacher Instruction by Content and Coverage.** In third-grade
classrooms in 2007, both Reading First and non-Reading First students
spent about one third of their time on comprehension (31 percent and 32
percent, respectively), as shown in Table 14. However, observers
documented a greater emphasis on phonics and fluency in Reading First
classrooms relative to non-Reading First group classrooms (for phonics,
14 percent in Reading First classrooms, compared with 4 percent in
non-Reading First classrooms; fluency:12 percent and 5 percent,
respectively). Non-Reading First classrooms were more often engaged in
vocabulary work (17 percent Reading First compared with 28 percent
non-Reading First).

Table 14. Content Focus (Percentage of Minutes Observed)\
by Grade and Treatment Group

+----------------------+-----------+-----------+-----------+-----------+
| **Content Focus**    | **2006    |           | **2007    |           |
|                      | Second    |           | Third     |           |
|                      | Grade**   |           | Grade**   |           |
+----------------------+-----------+-----------+-----------+-----------+
|                      | **RF**    | **Comp**  | **RF**    | **Comp**  |
|                      |           |           |           |           |
|                      | **Min =   | **Min =   | **Min     | **Min =   |
|                      | 885**     | 781.5**   | 579.5**   | 574**     |
+----------------------+-----------+-----------+-----------+-----------+
| Phonemic awareness   | 0.3%      | 0.0%      | 0.5%      | 2.8%      |
+----------------------+-----------+-----------+-----------+-----------+
| Phonics              | 1.6%      | 5.0%      | 14.0%     | 4.4%      |
+----------------------+-----------+-----------+-----------+-----------+
| Word recognition     | 0.0%      | 4.7%      | 6.4%      | 5.9%      |
+----------------------+-----------+-----------+-----------+-----------+
| Comprehension        | 46.2%     | 40.1%     | 31.2%     | 32.4%     |
+----------------------+-----------+-----------+-----------+-----------+
| Vocabulary           | 18.4%     | 18.0%     | 17.2%     | 28.7%     |
+----------------------+-----------+-----------+-----------+-----------+
| Reading connected    | 27.4%     | 30.4%     | 18.3%     | 21.1%     |
| text (not            |           |           |           |           |
| turn-taking)         |           |           |           |           |
+----------------------+-----------+-----------+-----------+-----------+
| Fluency              | 6.0%      | 1.8%      | 12.4%     | 4.7%      |
+----------------------+-----------+-----------+-----------+-----------+

**Figure 5. 2007 Content Focus by Treatment Group**

![](media/image6.png){width="6.495833333333334in"
height="2.870833333333333in"}

**Figure 6. 2006 Content Focus by Treatment Group**

![](media/image7.png){width="6.4944444444444445in"
height="2.8743055555555554in"}

Within comprehension, reading connected text, fluency, and vocabulary,
analyses show differences in teachers' instructional activities (see
Table 15). Most notably, when the focus was on comprehension, fluency,
or vocabulary, Reading First teachers tended to spend less time telling
or giving information to students than non-Reading First school teachers
did. In addition, when the focus was on comprehension, Reading First
teachers spent more time reading aloud (8 percent Reading First versus 1
percent non-Reading First). When the focus was on fluency instruction,
Reading First teachers were more likely to assess their students than
non-Reading First teachers (14 percent for Reading First teachers
compared with 0 percent for non-Reading First teachers).

Table 15. Third-Grade Teacher Activity by Content Focus\
(Percentage of Observational Segments)

  ---------------------- ------------------- ---------- ----------- ---------- ------------- ---------- ---------------- ----------
  **Teacher Activity**   **Comprehension**              **Reading              **Fluency**              **Vocabulary**   
                                                        Connected                                                        
                                                        Text**                                                           

                         **RF**              **Comp**   **RF**      **Comp**   **RF**        **Comp**   **RF**           **Comp**

  Assessment             0.0%                0.0%       0.0%        0.0%       13.6%         0.0%       0.0%             0.0%

  Coaching/scaffolding   4.8%                4.3%       5.1%        2.3%       4.5%          0.0%       7.0%             2.0%

  Check work             1.4%                3.6%       1.3%        1.6%       0.0%          0.0%       0.0%             8.2%

  Discussion             1.4%                0.7%       0.6%        0.0%       0.0%          0.0%       1.8%             2.0%

  Listening/watching     10.2%               10.1%      15.8%       25.0%      59.1%         57.1%      19.3%            12.2%

  Modeling               2.7%                2.9%       3.8%        3.1%       4.5%          0.0%       0.0%             0.0%

  Higher order           15.6%               19.4%      2.5%        6.3%       0.0%          0.0%       8.8%             2.0%
  questioning                                                                                                            

  Lower order            26.5%               28.1%      15.2%       15.6%      0.0%          0.0%       24.6%            16.3%
  questioning                                                                                                            

  Reading aloud          7.5%                0.7%       19.0%       23.4%      4.5%          14.3%      3.5%             0.0%

  Recitation             10.9%               9.4%       8.2%        3.1%       0.0%          0.0%       14.0%            16.3%

  Tell/give information  11.6%               20.1%      10.8%       10.2%      13.6%         28.6%      17.5%            34.7%

  None                   1.4%                0.0%       3.2%        6.3%       0.0%          0.0%       3.5%             0.0%

  Other                  6.1%                0.7%       14.6%       3.1%       0.0%          0.0%       0.0%             6.1%
  ---------------------- ------------------- ---------- ----------- ---------- ------------- ---------- ---------------- ----------

**Perceptions of Instructional Change**

The intent of the Wisconsin Reading First initiative is to provide
educators with the resources and support they need to effect change in
classroom instruction that will result in improved student outcomes.
Reading First classroom teachers, principals, and Reading First
coordinators reflected in interviews on changes that they attribute to
their involvement in Reading First. Non-Reading First classroom
teachers, principals, and literacy staff also were asked to comment on
various aspects of instructional improvement. For the most part, it
appears that instruction in Reading First classrooms is being positively
affected by the initiative.

Most (85 percent) of the teachers, principals, and Reading First
coordinators agreed that Reading First has resulted in positive changes
to their literacy instruction. Although a few teachers were not certain
of the effects of Reading First on instruction, only two teachers stated
that their school's literacy instruction had not changed since the
initial implementation of Reading First. Staff members discussed the use
of literacy centers, the five essential components of reading, the
90-minute reading block, leveled readers, differentiated instruction,
and use of assessments as improvements to their literacy instruction
resulting from Reading First.

The online survey confirms the results of the interviews. When given
several possible options in the online survey as to ways the Reading
First initiative may have changed their reading instruction during the
past school year, almost all Reading First teachers (93 percent)
indicated that it has changed their reading instruction in some manner.
More teachers (41 percent) indicated that the Reading First initiative
changed their instruction by enhancing general education instruction for
all children in the regular education classroom than any of the other
possible choices (see Table 16).

Table 16. Perceived Impact of Reading First on Classroom Instruction

+-------------------------------------------------------------+--------+
| Reading First Teachers                                      | Fre    |
|                                                             | quency |
|                                                             |        |
|                                                             | *N* =  |
|                                                             | 798    |
+-------------------------------------------------------------+--------+
| Enhances general education instruction for all children in  | 40.7%  |
| the regular education classroom.                            |        |
+-------------------------------------------------------------+--------+
| Provides more intense intervention for select children in   | 27.3%  |
| general education setting (e.g., paired reading, small      |        |
| groups).                                                    |        |
+-------------------------------------------------------------+--------+
| Provides more intense intervention for select children in   | 11.7%  |
| pull-out setting (e.g., special tutoring outside the        |        |
| regular classroom).                                         |        |
+-------------------------------------------------------------+--------+
| Enhances special-needs instruction for children with        | 5.1%   |
| disabilities or limited English proficiency in the regular  |        |
| classroom.                                                  |        |
+-------------------------------------------------------------+--------+
| Enhances special-needs instruction for children with        | 7.9%   |
| disabilities or limited English proficiency in special      |        |
| instructional settings outside the regular classroom.       |        |
+-------------------------------------------------------------+--------+
| No change in reading instruction.                           | 7.3%   |
+-------------------------------------------------------------+--------+

In the survey, teachers at Reading First schools were given a list of a
variety of different teaching skills and were asked to select the extent
to which the Reading First initiative helped them develop each of them
(see Table 17). The Reading First initiative was rated as moderately
helpful in building each of these skills by between 43 percent and 46
percent of teachers. In addition, 72 percent to 85 percent of teachers
indicated that the Reading First initiative was either moderately or
extremely helpful in all of the possible areas. Teachers were most
likely to say that the Reading First was extremely helpful in increasing
their knowledge about effective reading instruction practices and in
improving the way they teach reading.

Table 17. Perceived Impact of Reading First on Teacher Practice

  --------------------- ------------ ------------- ---------- ------------ -----------
  Reading First         Not          Applicable,   Somewhat   Moderately   Extremely
  Teachers              Applicable   but Not       Helpful    Helpful      Helpful
                                     Helpful at                            
                                     All                                   

  Become a better       1.7%         1.4%          12.9%      45.8%        38.2%
  teacher of reading                                                       
  skills\                                                                  
  (*n* = 806)                                                              

  Change my classroom   2.9%         2.5%          14.8%      45.8%        34.1%
  practices for the                                                        
  better\                                                                  
  (*n* = 806)                                                              

  Increase my knowledge 0.9%         1.6%          13.2%      45.1%        39.2%
  about effective                                                          
  reading instruction                                                      
  practices\                                                               
  (*n* = 803)                                                              

  Prepare better        3.0%         5.3%          17.8%      45.1%        28.8%
  reading plans**\                                                         
  **(*n* = 805)                                                            

  Assess student        2.4%         3.0%          18.1%      44.5%        32.1%
  reading needs to                                                         
  better plan                                                              
  instruction\                                                             
  (*n* = 803)                                                              

  Select and assign     3.5%         4.3%          20.0%      42.7%        29.5%
  more appropriate                                                         
  reading materials for                                                    
  my students\                                                             
  (*n* = 806)                                                              

  Expand my teaching    3.0%         2.5%          15.6%      45.7%        33.3%
  strategies to reach                                                      
  all students in my                                                       
  class\                                                                   
  (*n* = 806)                                                              

  Improve the way I     2.1%         2.4%          12.5%      43.3%        39.7%
  teach reading\                                                           
  (*n* = 801)                                                              
  --------------------- ------------ ------------- ---------- ------------ -----------

Teachers from Reading First schools were asked in the online survey to
select from a variety of possible ways that the Reading First initiative
may have increased their instructional collaboration, if at all. More
than half (53 percent) of the teachers indicated that there is now more
collaboration among all teachers, while 29 percent said there is more
collaboration with the general education teachers (see Table D36 in
Appendix D).

Overall, the literacy coaching described by the Reading First
coordinators in interviews is consistent with the model described by
Meyer, Camburn, Kimball, Milanowski, Morgan, Lowenhaupt, Schomisch, and
Kim (2006). Their report states that:

> literacy coaching is a relatively new intervention, aimed at improving
> the instructional capacity of content teachers, so that they can
> provide better literacy instruction for all students rather than
> depending on specialists and pull-out teachers for students struggling
> with literacy (Hall, 2004, as cited in Meyer et al., 2006, p.3).

The study cites empirical evidence that coaching can have a positive
effect on student achievement: "Adding observation, feedback, and
coaching to theory, demonstration and practice have been found to
increase teacher skill development and improve the transfer of training
to teachers' daily instructional practice" (Joyce and Showers, 1995 as
cited in Meyer et al., 2006, p.4). Almost all (93 percent) of the
Reading First teachers indicated that they implement the suggestions
they receive from their literacy coach and said that the suggestions
have worked to improve their instruction.

Reading First teachers expressed their experiences with coaching as
follows:

-   "I work a lot with \[the literacy coach\] and the other teachers.
    This is the first year I've been in a Reading First school. There is
    excellent support, and it was easy to pick it up and get it all
    organized. We focus on things from the basic training and the goal
    charts to fluency and ways to improve it."

-   "It \[coaching\] gives you new ideas, and an outsider can notice
    things you don't notice yourself."

-   "Yes, \[the literacy coach\] was the one who modeled it for me. When
    I first took over this Reading Mastery 3 group, she was the one who
    was modeling the pacing and going back and all of that....So that
    also is kind of nice for me to come back and see and she'll catch
    things and say better to do it this way."

Teachers were asked in the online survey about the extent to which their
students' skills improved as a result of the Reading First initiative
(see Table 18). More than half the Reading First teachers indicated that
they saw moderate or extreme improvement in all of the listed literacy
areas, except spelling. Teachers reported that the Reading First
initiative resulted in the most improvement in their students'
confidence in reading, reading comprehension, and phonics.

Table 18. Perceived Impact of Reading First on Students' Abilities,
Attitudes, or Interests

  -------------------- ------------ ------------- ---------- ------------ -----------
  Reading First        Not          Applicable,   Somewhat   Moderately   Extremely
  Teachers             Applicable   but Not       Improved   Improved     Improved
                                    Improved at                           
                                    All                                   

  Decoding skills\     3.8%         1.9%          23.4%      47.6%        23.2%
  (*n* = 806)                                                             

  Phonemic awareness\  4.5%         2.5%          22.2%      45.7%        25.2%
  (*n* = 806)                                                             

  Phonics\             2.4%         2.5%          20.7%      48.4%        26.1%
  (*n* = 798)                                                             

  Vocabulary           2.0%         2.6%          23.0%      52.6%        19.9%
  development\                                                            
  (*n* = 806)                                                             

  Oral language        2.7%         3.9%          26.6%      48.8%        18.0%
  development\                                                            
  (*n* = 805)                                                             

  Interest in reading\ 1.7%         4.3%          22.5%      45.5%        25.9%
  (*n* = 806)                                                             

  Reading              25.9%        3.5%          21.1%      53.3%        20.0%
  comprehension\                                                          
  (*n* = 806)                                                             

  Confidence in        2.2%         3.5%          18.2%      46.8%        29.3%
  reading\                                                                
  (*n* = 806)                                                             

  Spelling\            10.1%        10.1%         34.7%      38.0%        10.5%
  (*n* = 802)                                                             
  -------------------- ------------ ------------- ---------- ------------ -----------

Teachers also were asked in the survey to select from a range of
response options relating to how much influence the Reading First
initiative had on improving their students' academic performance during
the past year (see Table 19). Approximately one third (35 percent) of
these teachers reported that Reading First helped all of their students
moderately or substantially, and almost half indicated that the Reading
First initiative helped many of their students moderately or
substantially. Overall, 40 percent of teachers said that Reading First
substantially improved the performance of at least a few of their
students, while nearly half (49 percent) indicated that Reading First
had been moderately helpful to at least a few students.

Table 19. Perceived Impact of Reading First\
on Student Academic Performance

  ---------------------------------------------------------- ------------
  Reading First Schools                                      Frequency\
                                                             *n* = 801

  Academic performance is generally the same for my          4.1%
  students.                                                  

  A few of my students have been helped in minor ways.       2.1%

  A few of my students have been helped in moderate ways.    6.1%

  A few of my students have been helped substantially.       2.7%

  Many of my students have been helped in minor ways.        2.9%

  Many of my students have been helped in moderate ways.     22.8%

  Many of my students have been helped substantially.        24.5%

  All of my students have been helped in minor ways.         2.4%

  All of my students have been helped in moderate ways.      19.9%

  All of my students have been helped substantially.         12.5%
  ---------------------------------------------------------- ------------

Analyses of responses to both the 2006 and 2007 online teacher surveys
show that kindergarten teachers reported that Reading First improved
their students' abilities, attitudes, and interests to a greater extent
than did first-, second-, or third-grade teachers (*F* = 11.3, p \<
0.0001). Figure 7 exhibits reported student improvement attributed to
Reading First.

**Figure 7. Reported Student Improvement Attributed to Reading First**

![](media/image8.wmf){width="6.010416666666667in"
height="4.197916666666667in"}

Most Reading First coordinators, principals, and teachers (90 percent)
in Reading First schools stated that teaching was more consistent across
classrooms as a result of the program. All Reading First coordinators
and all but one principal stated that teaching was more consistent
across classrooms. Similarly, 86 percent of teachers reported that
teaching had become more consistent. One of the teachers stated that:

> \[W\]e communicate more; we talk about kids a lot more. We talk about
> how to help kids....I feel like I know what the kids are doing in
> first grade and what they're going to need when they come to second
> grade more than I did before. I also know what to help the kids with
> in second grade to make them more ready for third grade.

Another teacher asserted that:

> I believe we are focused on those five areas of phonemic awareness,
> phonics, vocabulary, fluency, and comprehension much more consistently
> than we had before in Reading First. It's kind of opened up a new
> communication between classrooms and consistent approach toward the
> priorities.

One Reading First principal noted that:

> Before Reading First, everybody did their own thing. \[Now\]
> everybody's on the same page. Our teachers have a common planning time
> everyday, for their reading. And you can go from one first grade to
> the other first grade, to the third one, and they're all doing the
> same thing. They're all in the same content area. It's remarkable.

One of the Reading First coordinators noted that a positive result of
the Reading First program is that:

> our teachers are finally communicating with each other. They meet at
> the beginning of the year and look over the previous intervention
> plans and talk to the next year's teachers about what they've done.
> There's a lot of idea sharing, a lot of support between the teachers,
> a lot of "this really worked for me, maybe you should try it... . How
> do I help this child"... it's really a camaraderie between the
> teachers now.

In non-Reading First schools, literacy staff, principals, and teachers
generally reported less consistency across classrooms. Only 38 percent
of literacy staff stated that teaching was consistent across classrooms.
Approximately 46 percent of principals at non-Reading First schools
reported that teaching was consistent, while a slightly higher
percentage of teachers reported consistency across classrooms (59
percent).

When asked if teachers were given the opportunity to regularly observe
each other in order to provide feedback and grow professionally,
respondents in all schools tended to report that this did not regularly
occur. However, 42 percent of Reading First coordinators, 40 percent of
principals, and 23 percent of teachers in Reading First schools reported
that there is some teacher observation of other classrooms, although it
does not occur on a regular basis. A teacher from a Reading First school
noted that teacher observations of other teachers "\[did not occur\]
regularly, but it does happen on occasion." Overall, slightly more
respondents from Reading First schools than from non-Reading First
schools reported that teachers did get to observe other classrooms to
some extent (29 percent compared to 19 percent).

**Use of Assessments**

The use of valid and reliable screening, diagnostic, progress
monitoring, and outcome assessments is another key goal of the Wisconsin
Reading First program. The explicit intent is to provide program staff
with training on the administration and interpretation of these
assessments and on the use of assessment data to drive instructional
decision making in the classroom. Reading First classroom teachers,
principals, and Reading First coordinators provided information on their
use of assessments to monitor student progress in reading. Non-Reading
First classroom teachers, principals, and literacy staff also were asked
to comment on the use of assessments.

**Use of Assessment Data for Instruction and Classroom Management.**
Principals both from Reading First and non-Reading First schools were
asked to rate how well they believed teachers were using assessments to
inform instructional decision making, to provide informed interventions
for struggling students, and to set up and manage classrooms. Sixty-four
percent of Reading First principals reported that teachers were using
assessments very well to inform instruction and to provide informed
interventions, while the remaining 36 percent said they were using
assessments somewhat well for the same purposes. Fifty-five percent
believed they were also using assessments very well to set up and manage
classrooms, and 45 percent said they were doing it somewhat well.

A majority of principals from non-Reading First schools (73 percent)
also believed that their teachers were using assessments very well to
set up and manage classrooms. When it came to providing informed
interventions for struggling students, 64 percent of principals said
teachers were using assessments very well. Slightly less than half of
these principals (45 percent) reported that teachers were using the
assessments very well to inform instructional decision making, while a
little more than half (55 percent) said they were doing it somewhat
well.

Although literacy staff and teachers were not asked to rate how well
they believed they were using assessments, they were asked to specify
how they used the assessments for the purposes already mentioned.
Reading First teachers and Reading First coordinators stated that when
it came to informing instructional decision making, they most often used
assessments to plan instruction (69 percent), group students (39
percent), and determine interventions (20 percent). A majority (80
percent) of non-Reading First teachers and literacy staff also reported
using assessments to plan instruction, but fewer respondents mentioned
using assessments to group students (23 percent). In providing informed
interventions for struggling students, Reading First coordinators and
teachers most frequently reported using assessments to determine what
specific areas they needed to reteach or remediate (43 percent), to
determine what materials and resources to provide to students (37
percent), and to group students by ability (33 percent). Non-Reading
First teachers, by contrast, said that they primarily use assessments to
determine what areas to reteach or remediate (77 percent). In terms of
setting up and managing classrooms, they often reported using
assessments to group students by ability (40 percent), to set up
appropriate centers (40 percent), and to determine who needed pullout
services or other individual attention (13 percent).

**Use of Student Data to Monitor Student Reading Progress.** Staff at
Reading First and non-Reading First schools overwhelmingly reported
using formal student assessments to monitor student progress. All the
Reading First coordinators and principals, and 96 percent of teachers
interviewed at Reading First schools said they monitor student reading
progress through formal student assessments. All of the literacy staff
and principals and 83 percent of teachers at non-Reading First schools
said they monitor student reading progress through formal assessments. A
majority of Reading First and non-Reading First staff (60 percent and 69
percent, respectively) also discussed using informal assessment to
monitor reading progress.

**Use of Assessment Data for Other Purposes.** Principals at Reading
First schools reported a somewhat different use of the assessment data
than did their respective school Reading First coordinators. Most
principals (82 percent) said they used this information to drive
instruction. Several principals (45 percent) also reported using the
information to target interventions and 36 percent of principals
reported using the information to monitor or assess teachers.

**Use of Reading First-Specific Assessment Tools.** Most of the teachers
who responded to the survey from Reading First schools indicated that
they use the Early Reading Diagnostic Assessment (ERDA), TerraNova, and
Dynamic Indicators of Basic Early Literacy Skills (DIBELS) in some
manner. Teachers most frequently reported using the ERDA as a diagnostic
tool (45 percent), the TerraNova to assess outcomes (57 percent), and
the DIBELS to monitor student progress (44 percent) or as a screening
tool (34 percent). A full breakdown of responses is presented in Table
20.

Table 20. Percent of Teachers Using Assessment Tools

to Measure Student Reading Proficiency

  -------------- ----------- --------------- --------------- ------------ ------------
  Reading First  **Not       **Screening**   **Diagnosis**   **Monitor    **Assess
  Schools        Using**                                     Progress**   Outcomes**

  ERDA\          6.3%        8.7%            45.0%           25.2%        14.8%
  (*n* = 793)                                                             

  TerraNova\     20.4%       2.8%            5.8%            14.2%        56.8%
  (*n* = 755)                                                             

  DIBELS\        2.7%        34.3%           7.7%            44.1%        11.1%
  (*n* = 801)                                                             
  -------------- ----------- --------------- --------------- ------------ ------------

In the non-Reading First schools, two thirds of teachers indicated they
do not take the DIBELS, and several other teachers did not respond to
the question asking about their use of the DIBELS. A minority of
non-Reading First teachers indicated that they the use the ERDA to
monitor student progress (25 percent) and the TerraNova to assess
outcomes (37 percent) or monitor progress (27 percent).

Perceived Impact of Assessments on Instruction. More than half the
teachers at Reading First schools reported in the survey that daily
student assessments (55 percent) and classroom performance assessments
(55 percent) have a sustained impact on classroom instruction (see Table
21). Almost half (49 percent) of the Reading First teachers reported
that diagnostic and outcome assessments have a moderate impact on
classroom instruction.

Table 21. Perceived Impact of Assessment Tools on Reading Instruction

  ------------- ---------------- ------------------- -------------- -------------
  Reading First Diagnostics and  Classroom           Daily Student  Progress
  Schools       Outcome          Performance         Assessments\   Monitoring\
                Assessment\      Assessments\        *n* = 794      *n* = 799
                *n* = 797        *n* = 794                          

  No impact     3.5%             2.1%                2.6%           1.8%

  Slight impact 19.4%            5.3%                6.0%           6.0%

  Moderate      49.4%            37.4%               36.0%          42.6%
  impact                                                            

  Sustained     27.6%            55.2%               55.3%          49.7%
  impact                                                            
  ------------- ---------------- ------------------- -------------- -------------

Teachers at non-Reading First schools most often reported that the daily
assessments, progress monitoring, and classroom performance assessments
each have a sustained impact on reading instructions (75 percent, 64
percent, and 61 percent, respectively).

**Proficiency in Using Reading First Assessment Materials.** Most
teachers at Reading First schools reported considering themselves
extremely or moderately proficient at using the Reading First assessment
materials (63 percent) and most teachers at non-Reading First schools\
(76 percent) reported that they consider themselves to be at least
moderately proficient at using reading assessment materials. Although
more non-Reading First teachers than Reading First-teachers noted they
felt proficient at using assessment, some of these assessment tools were
different, which could account for the differing perceptions.

**Usefulness of Reading First Assessment Tools.** At Reading First
schools, a large majority of the Reading First coordinators (75 percent)
and principals (82 percent) and 40 percent of the teachers said the
DIBELS was one of the most helpful tools for assessment. They frequently
commented that it is easy to administer and gives teachers immediate
feedback. As one coordinator explained, "I do like the DIBELS a lot.
Even though it's just a screening device, I think it presents some good
information that red-flags those kids we want red-flagged."

There was no single test that a majority of non-Reading First literacy
staff reported as being most useful. However, nearly half (47 percent)
of all interviewed non-Reading First staff reported relying on informal
testing or teacher observation.

In contrast to the popularity of the DIBELS among Reading First literacy
staff and principals, less than one tenth (9 percent) of the staff
members at non-Reading First schools mentioned the DIBELS as a useful
reading assessment.

**Level of Comfort With Reading First Assessments.** One of the
components of the Wisconsin Reading First program is the ERDA. The
Reading First coordinators were asked about how comfortable teachers
have become in administering this assessment over the course of the
grant. A majority of Reading First coordinators (83 percent) said that
it is their responsibility, not that of the classroom teachers, to
administer the ERDA. This was confirmed by many teachers (47 percent) at
Reading First schools who reported they do not administer the ERDA. In
the two schools where the Reading First coordinators said the teachers
do administer this assessment, the Reading First coordinators reported
that teachers are comfortable with its administration. However, several
teachers reported not being comfortable (18 percent) or only being
somewhat comfortable (29 percent) administrating this test to students.

Less than half the teachers (42 percent) reported they have become more
comfortable conducting the ERDA over time. The rest of the teachers
either were not asked if they have gotten more comfortable administering
the ERDA or they reported had not become more comfortable administering
it, often because they said they did not administer it.

**Review of Student Progress Data.** A majority of Reading First staff
(87 percent) said the data on student reading progress were reviewed by
the Reading First coordinators or Title 1 district coordinators,
teachers (85 percent), and the school administration (79 percent). Half
those individuals interviewed at Reading First schools said the data
were reviewed by everyone and/or by teams. A majority (75 percent) of
Reading First coordinators primarily reported that their principal's
role in monitoring student reading progress was to review the data.

A majority (75 percent) of non-Reading First literacy staff reported
that their principal meets with teachers or gives them feedback as a way
of monitoring instruction, 50 percent reported their principal reviews
the data, and 50 percent told interviewers that their principal is
highly involved in the monitoring process. A majority non-Reading First
staff (78 percent) said that teachers review the data, more than half
(55 percent) said that it is reviewed by the principals, and 37 percent
said that it is reviewed by the Reading First coordinators. Half the
staff reported that the data was reviewed by everyone or in teams.

A majority of interviewed Reading First (65 percent) and non-Reading
First (60 percent) staff members reported that data are reviewed at
multiple levels, by individual student, classroom, and by grade.
Slightly more than one quarter of the Reading First staff (27 percent)
and one third of non-Reading First staff (34 percent) look at the data
by individual student, while 27 percent of the Reading First staff and 9
percent of non-Reading First staff reported reviewing it by grade level.

**Program Implementation: Professional Development**

To what extent has professional development influenced change in
classroom reading instruction and the five essential components of
reading? What professional development have Reading First schools
received and is it of high quality, relevance, and utility?

Professional development for teachers, Reading First coordinators, and
principals is at the core of the Reading First initiative. The Wisconsin
Reading First proposal (2003) states, "The Wisconsin/INSIGHT
professional development program is designed to provide high quality
training in SBRR; essential components of reading instruction;
instructional strategies, programs and materials; \[and\] instructional
leadership training" (p. 81). As suggested by the statement in the
proposal, Wisconsin's Reading First professional development plan is
implemented in collaboration with INSIGHT, a division of
McGraw-Hill/Wright Group.

According to the professional development plan, Reading First teachers
are required to have Reading First basic training and they also receive
regular support via site visits to their school by INSIGHT, the
collaborating professional development provider for the state of
Wisconsin.

In interviews, Reading First classroom teachers (*n =* 45), principals
(*n =* 11), and Reading First coordinators (*n =* 12) shared their views
regarding the professional development they have received as a result of
their involvement in Reading First. Non-Reading First classroom teachers
(*n =* 49), principals (*n =* 11), and literacy staff (*n =* 8) also
were asked to comment on professional development related to literacy in
which they have participated. Overall, there are three primary findings
on professional development:

-   Most Reading First teachers, principals, and coordinators found the
    professional development to be useful and beneficial, and noted that
    it provided valued information on the five essential components,
    best practices, and differentiating instruction to meet student
    needs.

-   More than half (55 percent) the Reading First teachers reported that
    that their instruction had been observed and been given feedback on
    by INSIGHT, and, of them, all reported valuing the feedback and the
    ongoing relationship they had with the representative.

-   Professional development for Reading First teachers was more likely
    to cover the five essential components, differentiated instruction,
    classroom management, and the use of literacy centers. Non-Reading
    First teachers were more likely to receive professional development
    in writing and assessments.

**Professional Development Provided by INSIGHT**

As might be expected, teachers reported receiving professional
development from the INSIGHT provider, the district, and the school. The
most frequently mentioned professional development provider mentioned by
Reading First teachers was INSIGHT (65 percent) followed by the state
and the district (37 percent for both). The school was the third most
frequently cited provider (28 percent). Non-Reading First teachers most
frequently reported receiving professional development from the district
(49 percent) and the school (31 percent), suggesting that they receive
far less instruction from the state and do not rely on the professional
development provided by INSIGHT, even though they are allowed to
participate in the Reading First professional development under the
requirements of the grant.

In Year 4, INSIGHT made regular site visits to Reading First schools and
provided feedback to help teachers improve their instructional practice.
The frequency of these visits varied among schools from weekly to
monthly. More than half (55 percent) of the Reading First teachers
reported that they were observed during INSIGHT visits to the school.
All teachers who were observed stated that they received suggestions for
improving their instructional practice that were helpful and useful. As
one teacher explained:

> She \[the INSIGHT representative\] had half-hour conferences after her
> presentation, but then she'd also ask how you were going to apply
> that.... So it isn't just like she comes in, presents, and leaves.
> It's an ongoing, developing kind of relationship we have with her.

**Professional Development Provided to the Reading First Coordinators**

In addition to the guidance offered by the external professional
development provider, INSIGHT, the Wisconsin Reading First program also
provides for a Reading First coordinator to support teachers at the
building level. This coordinator is specifically charged with helping
teachers develop strong, evidence-based skills for the delivery of
effective reading instruction. Before working with teachers, Reading
First coordinators receive training designed to equip them with skills
needed to support teachers. In Year 4 of the Wisconsin Reading First
program, coordinators continued to receive extensive professional
development on a variety of topics.

Reading First coordinators reported receiving substantially greater
hours of professional development than non-Reading First literacy staff
who might be considered their counterparts. Reading First coordinators
reported between 30 hours to 150 hours of professional development,
while the range in hours of professional development reported by
non-Reading First literacy staff was reported as between 3 hours and 40
hours (see Table 22).

**Table 22. Professional Development Hours Reported by Literacy Staff**

at Reading First and Non-Reading First Schools

  -------------------- ------------------------ -------------------------
  **Professional       **Reading First          **Non-Reading First
  Development Hours    Coordinators\            Literacy Staff\
  Reported**           (*n* = 11)**             (*n* = 8)**

  150 or more hours    18.2%                    0.0%

  100--149 hours       36.4%                    0.0%

  50--99 hours         18.2%                    25.0%

  30--49 hours         27.3%                    37.5%

  Less than 30 hours   0.0%                     37.5%
  -------------------- ------------------------ -------------------------

In the interviews, Reading First coordinators reported that the
professional development they received encompassed a wider range of
topics than were mentioned by non-Reading First literacy staff. Literacy
coaching and vocabulary were the professional development topics most
frequently mentioned by Reading First coordinators (55 percent for
both), followed by professional development related to assessments,
data, and testing (24 percent). The most frequently reported
professional development topic mentioned by the literacy staff at
non-Reading First schools was interventions/response to intervention.

Compared with non-Reading First literacy staff, Reading First
coordinators more often reported that Reading First principals (73
percent) and teachers (55 percent) often accompanied them to Reading
First training sessions. Of the non-Reading First literacy staff, 18
percent mentioned principals and 36 percent mentioned teachers in this
context.

In response to being asked how this year's (2006--07) training differs
from last year's, 36 percent of Reading First coordinators commented
that the training was about the same and mentioned that it was a
continuation of previous professional development. Ratings regarding the
adequacy of the professional development received were very similar for
both Reading First and non-Reading First literacy staff, with
approximately 64 percent of interviewees rating it as excellent. One
Reading First coordinator noted:

> I think every year it seems to get better and better because I think
> they base it on the feedback that they get from us, and our needs have
> changed. We also believe in our field that we're all learning a lot
> more, so we know the questions to ask and what to ask as far as
> professional development, so it's definitely improved.

**Professional Development for Principals**

Principals also received a variety of professional development and were
supportive of it.

Generally, principals indicated that information regarding best
practices (73 percent), as well as the exchange of ideas with colleagues
(27 percent), were the most valuable aspects of the Reading First
professional development they had received this year. For example,
regarding professional development provided under Reading First one
principal said,

> Just in professional development in general, the Reading First has
> been wonderful for my staff. A lot of the staff members had never left
> this building to go to anything. And then, Reading First had the
> mandatory professional development. Now they go, they're so excited
> about what they bring back. They're all eager to go....So to me,
> what's made this program is the professional development.

Another principal noted,

> Going back and talking with colleagues about some of the strategies
> they're using to implement Reading First, \[discussing\] what's been
> successful and not successful. I think that's been the most valuable,
> because you learn from everybody else's errors, to not make those same
> things happen in your school.

Slightly more than half the principals interviewed (55 percent) reported
that the Reading First professional development should be focused on
specific needs with more targeted support for principals (particularly
new principals) and that there should be more training and consistency
in training for teachers on specific topics. The principals who said
this were more likely to be new, with 64 percent having less than two
years in their current school. Comparatively, more than one third (36
percent) of non-Reading First principals expressed a similar desire for
professional development for principals that was either deeper or more
specifically targeted to their individual needs. The remaining Reading
First principals indicated that the professional development was fine as
it is and need not be changed.

**Teacher Participation in Professional Development**

Overall, teachers at Reading First schools were very satisfied with the
professional development provided. In interviews, the majority of
Reading First teachers (92 percent) and non-Reading First teachers (88
percent) indicated that during Year 4 they had participated in the
professional development that was offered. In many cases, teachers had
participated in several professional development opportunities. Only
three of 39 teachers indicated that they had not attended Reading First
conferences or participated in Reading First training academy activities
during the past school year. Although many Reading First professional
development opportunities are offered to teachers at non-Reading First
schools as well, none of the non-Reading First teachers who were
interviews reported attending any.

Reading First teachers who responded to the online survey most
frequently rated school-based in-service training provided by local
school personnel, conferences or workshops offered by the district, and
in-service training offered as part of regular staff meetings as being
moderately or extremely effective (72 percent, 70 percent, and 61
percent, respectively). The other forms of professional development were
rated as not applicable by a majority of teachers (between\
57 percent and 79 percent), which suggests that the teachers did not
participate in these types of professional development (see Table 23).

Table 23. Reading First Teacher Ratings of the Effectiveness of Reading
First\
Professional Development Received in 2006--07

  ------------------------------------------------------------------------------------------
  Reading First    Not Applicable Applicable, but Somewhat      Moderately     Extremely
  Schools                         Not Effective   Effective     Effective      Effective
                                  at All                                       
  ---------------- -------------- --------------- ------------- -------------- -------------
  School-based     6.6%           3.0%            18.6%         42.7%          29.2%
  in-service                                                                   
  provided by                                                                  
  local school                                                                 
  personnel\                                                                   
  (*n* = 839)                                                                  

  Conference or    19.6%          0.5%            10.9%         36.3%          32.7%
  workshop offered                                                             
  outside the                                                                  
  district\                                                                    
  (*n* = 841)                                                                  

  In-service as    9.9%           4.2%            25.1%         43.6%          17.2%
  part of regular                                                              
  staff meetings\                                                              
  (*n* = 841)                                                                  

  Forming          28.7%          3.5%            19.1%         35.9%          12.9%
  instructional                                                                
  committees based                                                             
  on content or                                                                
  grade levels\                                                                
  (*n* = 839)                                                                  

  In-service       56.6%          2.1%            14.8%         20.8%          5.7%
  provided by a                                                                
  publishing                                                                   
  company\                                                                     
  (*n* = 838)                                                                  

  Reading First    **Not          **Applicable,   **Somewhat    **Moderately   **Extremely
  Schools          Applicable**   but Not         Effective**   Effective**    Effective**
                                  Effective at                                 
                                  All**                                        

  Yearlong         61.5%          2.0%            10.4%         18.7%          7.3%
  workshops\                                                                   
  (*n* = 834)                                                                  

  Mentorship       71.0%          2.9%            8.1%          12.0%          6.0%
  programs\                                                                    
  (*n* = 832)                                                                  

  Voluntary study  64.8%          2.9%            9.4%          16.3%          6.6%
  groups developed                                                             
  by teachers or                                                               
  professional                                                                 
  organizations\                                                               
  (*n* = 839)                                                                  

  Graduate         61.8%          1.4%            8.8%          17.7%          10.3%
  courses\                                                                     
  (*n* = 832)                                                                  

  School visits to 62.2%          4.1%            10.8%         14.9%          8.0%
  observe                                                                      
  innovative                                                                   
  programs\                                                                    
  (*n* = 834)                                                                  

  Follow-up        10.9%          7.2%            23.0%         35.7%          23.1%
  support from                                                                 
  district Reading                                                             
  First                                                                        
  coordinators\                                                                
  (*n* = 834)                                                                  

  Reading First    27.5%          0.6%            12.4%         32.6%          26.9%
  basic training\                                                              
  (*n* = 840)                                                                  

  Literacy         18.6%          3.0%            17.0%         36.9%          24.6%
  centers\                                                                     
  (*n* = 830)                                                                  

  Classroom        51.7%          4.5%            12.9%         20.9%          9.9%
  management                                                                   
  workshop\                                                                    
  (*n* = 837)                                                                  

  Special needs    65.9%          3.8%            11.1%         12.3%          6.8%
  workshop\                                                                    
  (*n* = 836)                                                                  

  English language 76.7%          1.3%            8.5%          9.8%           3.6%
  learners                                                                     
  workshop\                                                                    
  (*n* = 833)                                                                  

  Reading First    79.3%          1.7%            5.8%          8.9%           4.3%
  coaches\                                                                     
  (*n* = 830)                                                                  
  ------------------------------------------------------------------------------------------

In the interviews, Reading First teachers (36 percent) more often than
non-Reading First teachers (14 percent), reported receiving professional
development that emphasized the five essential components of reading. In
addition, Reading First teachers were more likely than non-Reading First
teachers to identify having received professional development on the
topics of differentiated instruction, classroom management, and the use
of literacy centers. However, non-Reading First teachers more often
reported that they had received professional development in writing (18
percent) and in assessments (14 percent) than did Reading First teachers
(5 percent for both writing and assessment). (See Table 24.)

Table 24. Professional Development Topics Mentioned by Teachers

+------------------------------------+---------------+-----------------+
| ***N*'s Exclude Nonresponses and   | **Reading     | **Non Reading   |
| Teachers Who Said "No Professional | First         | First           |
| Development"**                     | Teachers**    | Teachers**      |
|                                    |               |                 |
|                                    | ***n* = 36**  | ***n* = 44**    |
+------------------------------------+---------------+-----------------+
| Individual components of reading   | 36.1%         | 13.6%           |
| (comprehension, fluency,           |               |                 |
| vocabulary, or word study)         |               |                 |
+------------------------------------+---------------+-----------------+
| Reading First components or        | 19.4%         | 0.0%            |
| Reading First basic training       |               |                 |
+------------------------------------+---------------+-----------------+
| Differentiated instruction (DI)    | 16.7%         | 9.1%            |
+------------------------------------+---------------+-----------------+
| Classroom management               | 13.8%         | 4.5%            |
+------------------------------------+---------------+-----------------+
| Literacy Centers, Learning         | 11.1%         | 45.5%           |
| Centers, or Literacy Stations      |               |                 |
+------------------------------------+---------------+-----------------+
| Assessments-CSI Maps-Running       | 5.6%          | 13.6%           |
| Records                            |               |                 |
+------------------------------------+---------------+-----------------+
| Writing                            | 5.6%          | 18.2%           |
+------------------------------------+---------------+-----------------+

Of note, some teachers reported less focus on the five essential
components during the 120-minute literacy block. Based on an analysis of
variance (see Figure 8) of scores from the 2006 to 2007 online teachers'
survey, Grade 3 teachers reported significantly less emphasis on the
five essential components than did Kindergarten, Grade 1, and Grade 2
teachers (*F* = 5.2, p \< 0.01). This difference is evident in Figure 8
across both years as the line for Grade 3 teachers is considerably lower
than the lines for the other three grades.

**Figure 8. Reported Extent to Which Professional Development Addressed\
the Five Essential Components of Reading**

![](media/image9.wmf){width="6.010416666666667in"
height="4.197916666666667in"}

Each of the five components was rated as being addressed either
adequately or in an exemplary manner by 89 percent to 93 percent of
survey participants, as shown in Table 25.

Table 25. Reading First Teachers Reports of the Adequacy of Reading
First\
Professional Development in Addressing the Five Components of Reading

  ---------------- ------------ ---------- ---------- ------------- ----------------
  Reading First    Phonemic     Phonics\   Fluency\   Vocabulary\   Comprehension\
  Schools          awareness\   *n* = 857  *n* = 856  *n* = 855     *n* = 856
                   *n* = 857                                        

  Not applicable   1.8%         2.0%       1.6%       1.3%          1.8%

  Applicable, but  1.4%         0.5%       0.7%       0.5%          0.1%
  does not address                                                  

  Addresses, but   4.6%         4.1%       7.8%       9.2%          8.5%
  not adequately                                                    

  Adequately       60.1%        60.2%      59.8%      59.4%         58.6%
  addresses                                                         

  Addresses in an  32.2%        33.3%      30.0%      29.6%         31.0%
  exemplary manner                                                  
  ---------------- ------------ ---------- ---------- ------------- ----------------

Most survey respondents also indicated that the professional development
was moderately to extremely helpful for each of the five components of
reading. Respondents rated the helpfulness of all five reading
components fairly consistently. Overall, professional development was
rated as moderately helpful in addressing each component by between 82
percent and 84 percent of Reading First teachers.

In the interviews, teachers noted as particularly valuable:

-   Visits by INSIGHT (25 percent)

-   Reading First conventions (21 percent)

-   Differentiated instruction (12 percent)

-   Reading First Basic training (12 percent)

Two thirds of the teachers at Reading First schools who participated in
the online survey indicated that the professional development they
received from Reading First adequately addressed their students'
literacy needs. Eighty-seven percent indicated that the Reading First
professional development addressed their students' needs either
adequately or in an exemplary manner.

Overall Reports of the Quality of Professional Development

Teachers who responded to the online survey rated the Reading First
professional development with respect to giving them the tools they need
to be effective teachers. In particular, Reading First professional
development was moderately or extremely helpful in:

-   Showing teachers how to adjust classroom instruction to meet the
    needs of students with diverse characteristics (69 percent)

-   Increasing their use of assessment data in planning instruction (69
    percent)

-   Increasing their enthusiasm about teaching (70 percent)

-   Increasing their collaboration with other teachers (71 percent)

-   Improving the way they teach reading (80 percent)

-   Improving their knowledge of the five essential components of
    reading (84 percent)

When Reading First teachers were asked what was not useful, two thirds
(65 percent) commented that all of the professional development they
participated in was useful and beneficial. In addition, the percentage
of Reading First teachers who stated that the professional development
they received had little influence on their instruction (2 percent) is
lower than the percentage of non-Reading First teachers who gave this
response (11 percent). Non-Reading First teachers (14 percent) more
often reported that interaction and collaboration among teachers was a
valuable aspect of their professional development than did Reading First
teachers\
(0 percent).

**Program Implementation: Support for Successful Implementation**

What factors have influenced successful Wisconsin Reading First
implementation? What program and organizational support and monitoring
do Reading First schools receive and to what extent will the program be
sustainable after the grant ends?

A strong level of support would be expected to play a critical role in
the successful implementation of Wisconsin Reading First. As such, the
evaluation examines the extent to which there has been strong support
for the program in Year 4, as well as ongoing support for the program
and sustainability after the initial grant period ends.

In order to explore these issues, Reading First coordinators ( *n=* 12),
classroom teachers\
(*n =* 45), and principals (*n =* 11) were asked in interviews to
reflect on the overall level of support they received for Reading First
from the district, the overall level of support they received at the
school level, the type and quality of communication about the program,
monitoring of the program, major changes within their school or district
that would affect implementation of Reading First, and whether the
program would be sustainable after the grant ends. In order to see if
any of the issues of support were similar for Non-Reading First schools,
classroom teachers (*n =* 49), principals (*n =* 11), and literacy staff
(*n =* 8) also were asked to comment on similar aspects of literacy
support in their schools and districts as well as classroom-level
monitoring of literacy instruction. Because there is no comparable issue
of sustainability, Non-Reading first schools were not asked about that
issue. Following are the key findings:

-   Both Reading First and non-Reading First schools reported that
    district support of school literacy programs is strong.

-   Both Reading First and non-Reading First respondents reported that
    professional development was a primary support for teachers.

-   Principals at Reading First schools and non-Reading First schools
    support literacy programming through open lines of communication.

-   Although Reading First coordinators work directly with individual
    teachers to support their literacy instruction, literacy staff at
    non-Reading First schools tend to work with teachers in groups
    rather than individually.

-   Reading First coordinators and principals consistently monitor
    classroom level implementation, in contrast to their non-Reading
    First counterparts.

-   Reading First staff reported that the Reading First program model is
    sustainable and that it was most important to retain the focus on
    the five essential components of reading.

-   Funding for professional development, coordinators, and materials
    and the likelihood of losing staff who have developed expertise were
    the barriers to sustaining the Reading First program that were cited
    most frequently by Reading First staff.

**District Support for Schools**

A majority of the faculty at both Reading First (72 percent) and
non-Reading First schools (71 percent) characterized the district
support for the school as strong overall. Teachers made comments such
as:

> I think they've done an excellent job. The resources that are there,
> and they give us input as to what we need to build our program. Our
> literacy coach helps also, and the funds are used very appropriately
> for training and for materials and things that we need to enrich our
> reading programs.

A majority of teachers, principals, and coordinators at Reading First
schools (74 percent) agreed that the support from the district was very
important to the success of the Reading First program. As one principal
at a Reading First school put it: "It's very important. Without their
support and recognition of the reading process, and what all goes into
that beginning reading process, without that, you would not have a
strong reading program."

Respondents at Reading First schools reported stronger lines of
communication both within the school and between the school and district
administration, especially compared to non-Reading First schools. Within
the school, half of Reading First coordinators and slightly less than
half (42 percent) of the teachers at Reading First schools reported
communicating directly to the principal or higher administration about
literacy instruction. This is slightly more than was reported at
non-Reading First schools where 38 percent of literacy staff and 29
percent of teachers reported discussing the program with their
principals or upper administration. Similarly a larger percentage (46
percent) of Reading First staff than non-Reading First staff (35
percent) reported having in-school meetings to discuss their literacy
program.

Between the school and the district, slightly more than half (55
percent) of the principals at Reading First schools reported talking
directly to the school board or district about the Reading First
program. Almost half (47 percent) of teachers at Reading First schools
reported that their Reading First coordinator was a liaison to the
district. Conversely, at the non-Reading First schools, none of the
principals said they discuss their literacy program with the
administration or school board.

**School-Level Support**

When asked to describe what supported teachers in their implementation
of their literacy program (either Reading First or another program),
principals at both Reading First and non-Reading First schools discussed
professional development for teachers and feedback and support from
colleagues. More Reading First principals (86 percent) than non-Reading
First principals (55 percent) cited professional development as a key
resource in helping teachers implement the program. More than one third
(36 percent) of principals from both types of schools also mentioned the
support and feedback from colleagues and other school staff and district
and administrative support. Non-Reading First principals actually were
slightly more likely to mention funding as an important support than
Reading First principals (36 percent and 27 percent, respectively).

As one Reading First principal noted:

> I think the teachers believe in Reading First...they believe that the
> program supports student learning...overall, the teachers have a
> confidence in the system and how things are designed...they've got a
> lot of supplemental materials, intervention materials. So they've got
> a lot of tools to use to help struggling readers.

Similar to non-Reading First schools, open lines of communication were
an often-cited and highly valued form of support at Reading First
schools, where a majority of staff frequently reported communicating
with each other about literacy via formal meetings (66 percent) as well
as informal discussions (57 percent). More than half (51 percent) of
Reading First staff also indicated that they communicate about literacy
instruction as needed or that the principal maintains an open door
policy and is consistently available to the literacy staff.

Non-Reading First staff also frequently reported formal meetings to
discuss student literacy\
(64 percent). Nearly three quarters (74 percent) of teachers and
literacy staff at non-Reading First schools reported that their
principal communicated directly with the staff about literacy
instruction by reviewing monitoring data and providing feedback. In
addition, a majority of non-Reading First staff (62 percent) reported
informally meeting with the principals (see Table 26).

Table 26. Types of Literacy Communication

  ------------------------- ---------------------- ----------------------
  **Type of Literacy        **Reading First (*n* = **Non-Reading First
  Communication**           68)**                  (*n* = 68)**

  Monitoring and direct     0%                     74%
  feedback                                         

  Formal meetings           66%                    64%

  Informal meetings         57%                    62%

  Open door/as needed       51%                    0%
  ------------------------- ---------------------- ----------------------

As described in the original grant proposal, the school principal's role
in Reading First is to carry out the statutory requirements and
directives associated with the program, and principals generally seemed
to see their role in this light. When asked specifically about their
role, most Reading First principals (91 percent) mentioned ensuring
compliance and implementation. General support for the program was the
second most frequent response (45 percent) from principals regarding
their role, followed by providing materials and resources (36 percent).
When asked if the principal's role with respect to Reading First is
defined at the state level, slightly more than half the Reading First
principals (55 percent) affirmed that the role is well defined at the
state level, while 18 percent of principals asserted that the role is
not well defined at the state level. The most frequent responses from
non-Reading First principals regarding their role in literacy
instruction included general support (64 percent), providing materials
(27 percent), conveying relevant information (27 percent), and observing
teachers (18 percent).

Similarly, the Wisconsin Reading First grant proposal prescribes
specific responsibilities for Reading First coordinators in providing
support for literacy instruction at the school level. Reading First
coordinators are responsible for working with the teachers through
coaching, modeling, and observation to make sure that the Reading First
program is being implemented correctly. For the most part, it appears
that the Reading First coordinators fulfill this role. When asked to
describe their role in supporting literacy instruction, a majority (91
percent) of Reading First coordinators reported holding one-on-one
conferences with teachers to offer feedback, instructional ideas, and
strategies to support literacy instruction.

Most (90 percent) of Reading First teachers confirmed that the Reading
First coordinator provides ongoing, active support for their literacy
instruction. In addition to individual conferences, Reading First
teachers reported that Reading First coordinators provide general
support, which includes modeling lessons (89 percent), assistance in
meeting specific student needs (46 percent), and providing resources,
materials, and information (14 percent). Observing teachers and
assisting in lesson planning were additional ways that Reading First
coordinators reported supporting teachers in their professional
development. Many principals and teachers commented that their school's
Reading First coordinator went above and beyond what was expected to
provide the teachers with the skills and resources needed to carry out
effective literacy instruction.

This individualized support at Reading First schools is in marked
contrast to what was reported by the literacy staff at non-Reading First
schools where a large majority (88 percent) of the literacy staff
reported that they did not work with individual teachers. This finding
is generally consistent with what non-Reading First teachers reported in
their interviews. Nearly two thirds of non-Reading First teachers
reported that they have not worked with a literacy coach or a lead
teacher during the 2006-07 school year. The reasons mentioned most
frequently were that they do not have the time or that it is not an
option for them. Several non-Reading First literacy staff mentioned that
instead of working individually with teachers, they work with groups of
teachers at grade-level meetings.

[]{.mark}

Reading First coordinators spent a great deal of their time providing
professional development, while literacy staff at non-Reading First
schools primarily administered assessments. Many Reading First
coordinators discussed organizing and presenting professional
development and conducting pullout or out-of-school reading programs (58
percent reported for each). Several (42 percent) Reading First
coordinators also mentioned reporting information to the state or
district, modeling for teachers (33 percent), and directly monitoring
student progress (33 percent) as other duties conducted in support of
the program. In contrast, the literacy staff at non-Reading First
schools most often (75 percent) reported testing students and rarely (13
percent) reported conducting professional development. None of the
literacy staff at non-Reading First schools talked about modeling
lessons.

Table 27. Reading First Coordinators and Non-Reading First School
Literacy Staff\
Reports of Activities

  --------------------------- -------- --------------------------- -------
  **Reading First                      **Non-Reading First         
  Coordinators (*n =* 12)**            Literacy Staff (*n* = 8)**  

  Organizing and presenting   58.3%    Organizing and presenting   12.5%
  professional development             professional development    

  Conducting pullout or out   58.3%    Conducting pullout or out   0.0%
  of school reading programs           of school reading programs  

  Modeling for teachers       33.3%    Modeling for teachers       0.0%

  Monitoring Student progress 33.3%    Monitoring Student progress 0.0%

  Testing students            0.0%     Testing students            75.0%
  --------------------------- -------- --------------------------- -------

**Monitoring of Literacy Instruction**

Reading First coordinators and principals reported being actively
involved in monitoring the implementation of the program at the
classroom level.

All the Reading First coordinators said they monitor the implementation
of the literacy program at the classroom level in some fashion.
Conversely, slightly more than one third (38 percent) of the literacy
staff at non-Reading First schools said they monitor the implementation
of the schools' literacy program at the classroom level.

Similarly, all of the Reading First principals interviewed reported
observing teachers' classroom instruction to monitor literacy
instruction at the classroom level, a statement that was confirmed by
Reading First coordinators. A majority of principals (55 percent) at
Reading First schools said that they visit each classroom daily, and a
majority of teachers at Reading First schools (80 percent) provided
further confirmation that their instruction is monitored via classroom
observation by their principal or Reading First coordinator. Several
teachers mentioned assessments (24 percent), the review of lesson plans
or class schedules (24 percent), and formal meetings (22 percent) as
methods commonly used to monitor their literacy instruction.

Literacy staff and principals at non-Reading First schools seemed to
perceive slightly different roles of the principal in terms of
monitoring literacy instruction at the classroom level. Although only 38
percent of non-Reading First literacy staff reported that their
principal observes teachers in their classroom and only one mentioned
that the principal reviews data, 81 percent of the principals at
non-Reading First schools reported that they visit the classrooms of
literacy teachers, 54 percent reported reviewing data, and 63 percent
noted that they visit each classroom daily. Just more than half (55
percent) of non-Reading First teachers mentioned that their classroom
instruction is monitored via direct observation. Non-Reading First
teachers most often (71 percent) reported that assessments are used to
monitor their literacy instruction.

Reading First coordinators appear to be more comfortable in their role
of monitoring classroom literacy instruction than their non-Reading
First peers. Most (83 percent) Reading First coordinators reported
feeling confident in their ability to monitor classroom literacy
instruction in their school. Only half the literacy staff at non-Reading
First schools indicated that they were equally confident in their
ability to monitor their teachers' literacy instruction.

Most (75 percent) Reading First coordinators said the data they get from
classroom observations is used to monitor the effectiveness of literacy
teachers, one third reported using data to track students, and another
third said they determine professional development needs based on what
they learn from monitoring literacy instruction.

In contrast to Reading First coordinators, none of the non-Reading First
literacy staff reported using observation forms to assess instruction.
Non-Reading First literacy staff most often reported that they use data
to make decisions about differentiation (63 percent), to discuss
struggling students (50 percent), or to assist literacy teachers through
classroom restructuring or offering new ideas on teaching methods (50
percent).

**Sustainability of Reading First**

A goal of Wisconsin Reading First is to have a lasting effect on
participating schools after the grant has ended.

The majority of teachers (63 percent), Reading First coordinators (55
percent), and principals (55 percent) reported that the Reading First
program model is sustainable at their school. A full breakdown of
responses is presented in Table 28.

Table 28. Perceived Sustainability of Reading First

  --------------------- --------------------- -------------- ------------ ----------
  **Do You Feel That    **Reading First       **Principal\   **Teacher\   **Total\
  the Reading First     Coordinator\          *n* = 11**     *n* = 43**   *N* = 66**
  Program Is            *n* = 12**                                        
  Sustainable in Your                                                     
  School?**                                                               

  Yes                   54.5%                 54.5%          62.7%        59.8%

  Somewhat              45.5 %                45.5%          25.6%        32.5%

  Don't know or hope so 0.0%                  0.0%           4.7%         3.0%

  No                    0.0%                  0.0%           4.7%         3.0%
  --------------------- --------------------- -------------- ------------ ----------

The component most cited by both teachers and principals (50 percent) as
being important to maintain when the current Reading First program ends
was the five components of reading. Assessment was the second most
frequently cited (38 percent), followed by the reading block (26
percent). See Table 29 for full breakdown of responses by role.

**Table 29. Core Components Mentioned As Important to Retain**

  --------------------- --------------------- -------------- ------------ ----------
  **Reading First**     **Reading First       **Principal\   **Teacher\   **Total\
                        Coordinator\          *n* = 11**     *n* = 43**   *N* = 66**
                        *n* = 12**                                        

  Five components       8.3%                  72.3%          55.8%        49.9%

  Assessments           54.5%                 72.3%          30.2%        41.6%

  Reading block         25.0%                 36.4%          23.2%        25.7%
  (90--120 min)                                                           
  --------------------- --------------------- -------------- ------------ ----------

With regard to barriers to sustaining the Reading First program, funding
and staffing were mentioned most frequently by Reading First respondents
(see Table 30).

Table 30. Perceived Barriers to Sustaining the Reading First Program

  ------------------------- ----------------- -------------- ------------ ----------
  **Reading First**         **Literacy Staff\ **Principal\   **Teacher\   **Total\
                            *n* = 12**        *n* = 11**     *n* = 43**   *N* = 66**

  Funding                   75.0%             72.7%          58.1%        63.6%

  Staffing                  41.6%             54.5%          41.9%        43.9%

  Lack of professional      33.3%             9.1%           16.3%        18.2%
  development                                                             
  ------------------------- ----------------- -------------- ------------ ----------

The following quotes reflect the views of school staff regarding
barriers and other issues related to sustainability.

> Money, I don't know what will happen when the grant is gone, if the
> district will keep a literacy coach in each of these schools. I'm sure
> that a lot of the conferences and workshops, that money will be gone
> and the workshops will dry up and the opportunities to go won't be as
> great. For example, the building of the libraries in the classrooms,
> that's all coming out of Reading First money and I can't see that
> money coming from the district either.
>
> It will be taxing on the school's finances to maintain the level of
> staffing.
>
> I see barriers as lack of funding and I think that when our current
> reading coordinator retires, which is this year, we're going to have
> to struggle having new leadership. And that's going to be an issue
> because she has a wealth of knowledge that's going with her. I see
> those as our two biggest barriers.
>
> The cost of having literacy coaches, INSIGHT people come in. The cost
> of having some of the workshops that we're able to go to and affording
> the subs or the travel or the motel accommodations, those things might
> not be as easily sustainable. The INSIGHT visits, those types of
> things probably would not be affordable.

**Wisconsin Reading First Year 4 Program Effectiveness**

A key component of measuring the effectiveness of the Wisconsin Reading
First program is the longitudinal examination of student achievement
data for several reading assessments. This section documents student
performance on three assessments and relates it to the evaluation
questions for Year 4 of the Wisconsin Reading First grant. The three
assessments discussed are the Wisconsin Knowledge and Concepts Exam
(WKCE, formerly the Wisconsin Reading Comprehension Test), the
TerraNova, and the Early Reading Diagnostic Assessment (ERDA). The
assessment approach for measuring the five essential components of
reading is shown in Table 31.

Table 31. Assessment Approach for Testing the Essential Components of
Reading

  --------------------------- -------------------------------------------
  **Component of Reading**    **Assessment**

  Phonemic awareness          ERDA Phonemes Test

  Phonics                     TerraNova Word Analysis Test

  Vocabulary                  TerraNova Vocabulary Test

  Fluency                     ERDA Passage Fluency Test

  Comprehension               TerraNova Reading Comprehension Test
  --------------------------- -------------------------------------------

The achievement study is comprised of two distinct substudies. The
first, the Progress Tracking Study, examines student achievement over
time for Reading First schools whereas the second, the Comparison Group
Study, compares growth of student achievement within Reading First
schools to growth in a similarly structured group of peers not in
Reading First schools. Key findings across these two studies are listed
below. A more detailed description of the analysis and results follows
this summary of findings.

**Reading First Schools Versus Statewide WKCE Performance**

-   The achievement gap on the WKCE between Reading First schools and
    the state has not changed over the course of program implementation
    (the percent of students from Reading First schools achieving
    proficiency or above on the assessment remained stable relative to
    the statewide percent).

-   The gain showed by African-American Reading First students from 2004
    to 2005 (moving from a gap of -17 percent to roughly -10 percent
    relative to statewide African-American students) was maintained from
    2005 to 2006.

-   After remaining relatively stable from 2004 to 2005, the relative
    percent difference between students with disabilities in Reading
    First schools and statewide worsened in 2006.

**Reading First Schools Versus Comparison Group Schools**

-   Reading First schools showed greater achievement at Grade 3 and
    higher growth rates than their counterparts in the Comparison Group
    on the TerraNova word analysis assessment (an indicator of Phonemic
    Awareness). However, no program impact was found for the TerraNova
    composite, reading, and vocabulary assessments.

-   Although Reading First schools made small gains in reading
    proficiency from 2004 to 2005, these schools lagged slightly behind
    schools in the comparison group in 2006 (in terms of percentage of
    students at or above proficiency on the WKCE).

In addition, a study of teacher survey responses indicated that schools
where teachers reported a higher impact of Reading First professional
development showed higher overall TerraNova performance. This result
suggests that teachers who are committed to the program and believe in
its effectiveness were more likely have students who showed greater
growth on the TerraNova assessment.

**Progress Tracking Study**

The Progress Tracking Study is guided by the following three evaluation
questions:

1.  What are the gains in reading proficiency for Wisconsin Reading
    First students, disaggregated by gender, racial/ethnic minority
    status, LEP status, disability status, economic status, and grade
    level?

2.  Do gains in reading proficiency vary with the extent to which the
    essential components of reading have been implemented?

3.  How do reading proficiency results for Wisconsin Reading First
    students compare with statewide reading proficiency results on the
    WKCE (formerly WRCT) at the end of Grade 3, disaggregated by gender,
    racial/ethnic minority status, LEP status, disability status, and
    economic status?

The first and third questions call for measuring gains in reading
proficiency broken down by demographic subgroup. Baseline measurements
for the assessments were established in the second year of the grant,
which was the first year of implementation and the first year teachers
utilized the Reading First assessment system. These data are analyzed
cross-sectionally because Grade 3 is the first year students are
required to take the WKCE (and, thus, students cannot be tracked
longitudinally). The analysis requires aggregating the WKCE data by
Wisconsin Reading First status and then performing subgroup analyses by
gender, racial/ethnic minority status, LEP status, disability status,
and economic status.

The second question was approached through a hierarchical linear model
(HLM) pairing implementation survey scale scores with longitudinal
student TerraNova data. These implementation scale scores were developed
from the surveys administered to teachers from both Reading First
schools and comparison group schools. As this question also is addressed
for the comparison group study, the analysis results will be presented
with the results from that study.

**Grade 3 Results on Wisconsin Knowledge and Concepts Exam (WKCE).**
Table 32 shows the distribution of Grade 3 students among the four
proficiency levels (i.e., minimal, basic, proficient, or advanced) for
the WKCE (fall 2005 and 2006) and WRCT (fall 2004). Note that
comparisons of the 2004 data to the data from 2005 and 2006 must be
interpreted cautiously because the achievement results are taken from
two separate assessments. The test changed at the conclusion of Year 2
of the grant and no equating methods were incorporated into the system.
Comparisons of the performance of Wisconsin Reading First students to
students statewide can best be made within a year rather than across
years. The tables include relative percent difference values (calculated
as the difference between the state and Reading First schools,
percentage proficient, divided by the state percent proficient). These
values show the relative difference between Reading First schools and
the statewide population. A negative value indicates that the Reading
First schools have fewer students performing at or above proficiency
than the statewide population and a positive value indicates the
opposite.

This evaluation focuses on the proficiency ratings for the 2006 test
administration; these are in the last two columns of the tables below.
In fall 2006 WKCE, 67 percent of Reading First students performed at or
above the proficient level compared with 82 percent statewide (a
relative percent difference of -18 percent). This difference in
performance is similar to what was seen in 2005 (19 percent) and 2004
(17 percent). It appears that the overall WKCE achievement gap between
Reading First schools and the state average has not changed over the
course of program implementation.

Table 32. Comparison of Reading First Schools and the Statewide
Population:\
Proficiency Distribution

+------------+--------+--------+--------+--------+--------+--------+
| **Grade    | **Fall |        | **Fall |        | **Fall |        |
| 3 **       | 2004   |        | 2005   |        | 2006   |        |
|            | WRCT** |        | WKCE** |        | WKCE** |        |
+------------+--------+--------+--------+--------+--------+--------+
|            | *      | **S    | *      | **S    | *      | **S    |
|            | *WRF** | tate** | *WRF** | tate** | *WRF** | tate** |
|            |        |        |        |        |        |        |
|            | ***N*  | ***N*  | ***N*  | ***N*  | ***N*  | ***N*  |
|            | =      | =      | =      | =      | =      | =      |
|            | 3      | 57     | 2      | 56     | 3      | 58     |
|            | ,113** | ,168** | ,942** | ,594** | ,181** | ,187** |
+------------+--------+--------+--------+--------+--------+--------+
|            | **Per  | **Per  | **Per  | **Per  | **Per  | **Per  |
|            | cent** | cent** | cent** | cent** | cent** | cent** |
+------------+--------+--------+--------+--------+--------+--------+
| Advanced   | 21.6%  | 42.9%  | 25.4%  | 43.4%  | 26.8%  | 45.3%  |
+------------+--------+--------+--------+--------+--------+--------+
| Proficient | 53.4%  | 47.9%  | 41.6%  | 38.9%  | 40.4%  | 36.6%  |
+------------+--------+--------+--------+--------+--------+--------+
| Basic      | 21.6%  | 8.1%   | 23.1%  | 13.8%  | 22.3%  | 13.6%  |
+------------+--------+--------+--------+--------+--------+--------+
| Minimal    | 3.5%   | 1.1%   | 9.9%   | 3.9%   | 10.5%  | 4.5%   |
+------------+--------+--------+--------+--------+--------+--------+
| Total      | 100.0% | 100.0% | 100.0% | 100.0% | 100.0% | 100.0% |
+------------+--------+--------+--------+--------+--------+--------+
| >          | 75.0%  | 90.8%  | 67.0%  | 82.3%  | 67.2%  | 81.9%  |
| Proficient |        |        |        |        |        |        |
| > or Above |        |        |        |        |        |        |
+------------+--------+--------+--------+--------+--------+--------+
| >          | -17.4% |        | -18.6% |        | -17.9% |        |
| Percentage |        |        |        |        |        |        |
| > Point    |        |        |        |        |        |        |
| >          |        |        |        |        |        |        |
| Difference |        |        |        |        |        |        |
+------------+--------+--------+--------+--------+--------+--------+

***Gender*.** Table 33 shows the proficiency distribution for males and
females, respectively, in the Wisconsin Reading First schools compared
with the statewide population. For the 2006 WKCE, 73 percent of female
students and 62 percent of male students from Reading First schools
performed at a level of proficient or above, compared to 85 percent and
79 percent statewide, respectively. The relative percent difference for
the Reading First schools compared to the state results was nearly -15
percent for the females and -21 percent for the males, indicating that
females in Reading First schools are performing better compared with
their counterparts at the state level than the males in Reading First
schools compared with their counterparts at the state level. Within
gender, the achievement difference between Reading First schools and the
state average seems to have remained unchanged over the course of
program implementation.

Table 33. Comparison of Reading First Schools and the Statewide
Population:\
Proficiency Distribution by Gender

<table>
<colgroup>
<col style="width: 26%" />
<col style="width: 11%" />
<col style="width: 12%" />
<col style="width: 0%" />
<col style="width: 12%" />
<col style="width: 0%" />
<col style="width: 12%" />
<col style="width: 0%" />
<col style="width: 12%" />
<col style="width: 0%" />
<col style="width: 12%" />
</colgroup>
<tbody>
<tr class="odd">
<td rowspan="2"><strong>Grade 3</strong></td>
<td colspan="2"><strong>Fall 2004 WRCT</strong></td>
<td colspan="4"><strong>Fall 2005 WKCE</strong></td>
<td colspan="4"><strong>Fall 2006 WKCE</strong></td>
</tr>
<tr class="even">
<td><p><strong>WRF</strong></p>
<p><strong><em>N</em> = 3,113</strong></p></td>
<td><p><strong>State</strong></p>
<p><strong><em>N</em> = 57,168</strong></p></td>
<td colspan="2"><p><strong>WRF</strong></p>
<p><strong><em>N</em> = 2,942</strong></p></td>
<td colspan="2"><p><strong>State</strong></p>
<p><strong><em>N</em> = 56,594</strong></p></td>
<td colspan="2"><p><strong>WRF</strong></p>
<p><strong><em>N</em> = 3,181</strong></p></td>
<td colspan="2"><p><strong>State</strong></p>
<p><strong><em>N</em> = 58,187</strong></p></td>
</tr>
<tr class="odd">
<td colspan="11"><strong>Female</strong></td>
</tr>
<tr class="even">
<td>Advanced</td>
<td>25.6%</td>
<td>46.9%</td>
<td colspan="2">28.4%</td>
<td colspan="2">47.2%</td>
<td colspan="2">28.7%</td>
<td colspan="2">49.1%</td>
</tr>
<tr class="odd">
<td>Proficient</td>
<td>52.3%</td>
<td>45.8%</td>
<td colspan="2">44.3%</td>
<td colspan="2">38.3%</td>
<td colspan="2">44.0%</td>
<td colspan="2">36.2%</td>
</tr>
<tr class="even">
<td>Basic</td>
<td>19.1%</td>
<td>6.3%</td>
<td colspan="2">20.5%</td>
<td colspan="2">11.8%</td>
<td colspan="2">19.9%</td>
<td colspan="2">11.5%</td>
</tr>
<tr class="odd">
<td>Minimal</td>
<td>3.0%</td>
<td>1.0%</td>
<td colspan="2">6.8%</td>
<td colspan="2">2.8%</td>
<td colspan="2">7.4%</td>
<td colspan="2">3.1%</td>
</tr>
<tr class="even">
<td>Proficient at or above</td>
<td>77.9%</td>
<td>92.7%</td>
<td colspan="2">72.7%</td>
<td colspan="2">85.5%</td>
<td colspan="2">72.7%</td>
<td colspan="2">85.4%</td>
</tr>
<tr class="odd">
<td>Percentage point change</td>
<td colspan="2">-16.0%</td>
<td colspan="4">-15.0%</td>
<td colspan="4">-14.9%</td>
</tr>
<tr class="even">
<td colspan="11"><strong>Male</strong></td>
</tr>
<tr class="odd">
<td>Advanced</td>
<td>17.9%</td>
<td colspan="2">39.6%</td>
<td colspan="2">22.4%</td>
<td colspan="2">39.8%</td>
<td colspan="2">25.1%</td>
<td>41.6%</td>
</tr>
<tr class="even">
<td>Proficient</td>
<td>54.3%</td>
<td colspan="2">50.0%</td>
<td colspan="2">39.0%</td>
<td colspan="2">39.4%</td>
<td colspan="2">37.0%</td>
<td>36.9%</td>
</tr>
<tr class="odd">
<td>Basic</td>
<td>23.9%</td>
<td colspan="2">9.4%</td>
<td colspan="2">25.7%</td>
<td colspan="2">15.7%</td>
<td colspan="2">24.5%</td>
<td>15.5%</td>
</tr>
<tr class="even">
<td>Minimal</td>
<td>3.9%</td>
<td colspan="2">1.0%</td>
<td colspan="2">12.8%</td>
<td colspan="2">5.1%</td>
<td colspan="2">13.4%</td>
<td>5.9%</td>
</tr>
<tr class="odd">
<td>Proficient at or above</td>
<td>72.2%</td>
<td colspan="2">89.6%</td>
<td colspan="2">61.4%</td>
<td colspan="2">79.2%</td>
<td colspan="2">62.1%</td>
<td>78.5%</td>
</tr>
<tr class="even">
<td>Percentage point change</td>
<td colspan="3">-19.4%</td>
<td colspan="4">-22.5%</td>
<td colspan="3">-20.9%</td>
</tr>
</tbody>
</table>

***Ethnicity*.** Table 34 compares the percent proficient by ethnicity
for the Wisconsin Reading First schools with the percent proficient for
the statewide populations. The fall 2006 relative percent differences
for each group are as follows: Native American (15 percent), Asian (-16
percent), African American (-10 percent), Hispanic (-8 percent) and
Caucasian (-7 percent). For African-American, Hispanic, and Caucasian
Reading First students, the achievement differences were very similar to
those seen in 2005. The gain showed by African-American Reading First
students from 2004 to 2005 (moving from a gap of 17 percent to roughly
10 percent) was maintained from 2005 to 2006. The gain in achievement
shown by Asian Reading First students from 2004 to 2005 (from nearly -16
percent to -7 percent) disappeared in 2006 as the achievement gap
returned to roughly -16 percent. Native American Reading First students
showed the largest gains compared to the state average, moving from a
relative percent difference of 2 percent in 2004 and 2005 to a
difference of 15 percent in 2006.

[]{.mark}

Table 34. Comparison of Reading First Schools and the Statewide
Population:\
Proficiency Distribution by Ethnicity

  --------------------- --------- ---------- --------- ---------- --------- ----------
  **Grade 3**           **Fall               **Fall               **Fall    
                        2004                 2005                 2006      
                        WRCT**               WKCE**               WKCE**    

                        **WRF\    **State\   **WRF\    **State\   **WRF\    **State\
                        *N* =     *N* =      *N* =     *N* =      *N* =     *N* =
                        3,113**   57,168**   2,942**   56,594**   3,181**   58,187**

  **Native American**                                                       

  Advanced              25.8%     28.1%      40.9%     25.3%      55.0%     30.0%

  Proficient            61.3%     57.3%      36.4%     50.5%      30.0%     43.7%

  Basic                 12.9%     12.5%      13.6%     19.0%      10.0%     19.9%

  Minimal               0.0%      2.1%       9.1%      5.1%       5.0%      6.4%

  Proficient or Above   87.1%     85.4%      77.3%     75.8%      85.0%     73.7%

  Percentage Point      2.0%                 2.0%                 15.4%     
  Difference                                                                

  **Asian**                                                                 

  Advanced              18.6%     36.3%      23.2%     37.4%      24.8%     36.6%

  Proficient            56.5%     52.7%      51.2%     42.7%      40.7%     41.2%

  Basic                 22.4%     9.9%       24.0%     16.6%      25.7%     17.5%

  Minimal               2.5%      1.1%       1.6%      3.3%       8.8%      4.8%

  Proficient or Above   75.1%     89.0%      74.4%     80.1%      65.5%     77.8%

  Percentage Point      -15.6%               -7.1%                -15.8%    
  Difference                                                                

  **African American**                                                      

  Advanced              8.1%      15.6%      10.6%     15.8%      13.0%     19.5%

  Proficient            51.3%     56.3%      41.7%     42.9%      41.6%     41.3%

  Basic                 34.9%     24.0%      32.6%     29.9%      28.1%     26.7%

  Minimal               5.7%      4.2%       15.1%     11.4%      17.3%     12.5%

  Proficient or Above   59.4%     71.9%      52.3%     58.7%      54.6%     60.8%

  Percentage Point      -17.4%               -10.9%               -10.2%    
  Difference                                                                

  **Hispanic**                                                              

  Advanced              17.2%     26.3%      21.5%     23.5%      19.3%     22.9%

  Proficient            63.1%     61.3%      45.7%     49.6%      42.5%     44.1%

  Basic                 16.4%     11.3%      24.9%     21.4%      28.5%     24.3%

  Minimal               3.3%      1.3%       7.8%      5.5%       9.7%      8.7%

  Proficient or Above   80.3%     87.5%      67.2%     73.0%      61.8%     67.0%

  Percentage Point      -8.2%                -7.9%                -7.7%     
  Difference                                                                
  --------------------- --------- ---------- --------- ---------- --------- ----------

  --------------------- -------- -------- -------- -------- -------- ----------
  **Grade 3**           **Fall            **Fall            **Fall   
                        2004              2005              2006     
                        WRCT**            WKCE**            WKCE**   

                        **Fall   **Fall   **Fall   **Fall   **Fall   **State\
                        2004     2005     2006     2004     2005     *N* =
                        WRCT**   WKCE**   WKCE**   WRCT**   WKCE**   58,187**

  **Caucasian**                                                      

  Advanced              35.8%    49.0%    41.4%    49.5%    42.2%    52.0%

  Proficient            52.8%    44.9%    39.8%    37.0%    38.6%    34.8%

  Basic                 9.8%     5.1%     13.0%    10.7%    14.5%    10.3%

  Minimal               1.6%     1.0%     5.8%     2.8%     4.7%     2.9%

  Proficient or Above   88.6%    93.9%    81.2%    86.5%    80.8%    86.8%

  Percentage Point      -5.6%             -6.1%             -6.9%    
  Difference                                                         
  --------------------- -------- -------- -------- -------- -------- ----------

***English Proficiency*.** Table 35 compares the proficiency of Reading
First students with limited English proficiency (LEP) with their
statewide counterparts. In 2005, the Reading First LEP students scored
better than their statewide counterparts by 4 percent. However, in 2006
the relative percent difference dropped to nearly -9 percent, indicating
that these students are once again performing below the state average.
In 2006, approximately 56 percent of Reading First LEP students were
proficient or above compared with 62 percent statewide.

Table 35. Comparison of Reading First Schools and the Statewide
Population:\
Proficiency Distribution by English Proficiency

  ---------------------- --------- ---------- --------- ---------- --------- ----------
  **Grade 3**            **Fall               **Fall               **Fall    
                         2004                 2005                 2006      
                         WRCT**               WKCE**               WKCE**    

                         **WRF\    **State\   **WRF\    **State\   **WRF\    **State\
                         *N* =     *N* =      *N* =     *N* =      *N* =     *N* =
                         3,113**   57,168**   2,942**   56,594**   3,181**   58,187**

  **Limited English                                                          
  Proficient**                                                               

  Advanced               13.3%     20.0%      18.5%     16.4%      12.5%     15.1%

  Proficient             58.0%     64.3%      54.3%     53.4%      43.9%     46.8%

  Basic                  25.2%     12.9%      23.9%     24.9%      32.8%     28.1%

  Minimal                3.4%      2.9%       3.3%      5.2%       10.7%     10.1%

  Proficient or Above    71.3%     84.3%      72.8%     69.8%      56.4%     61.8%

  Percentage Point       -15.4%               4.2%                 -8.8%     
  Difference                                                                 

  **English Proficient**                                                     

  Advanced               23.0%     43.9%      25.9%     44.7%      28.5%     47.3%

  Proficient             52.6%     46.9%      40.8%     38.2%      40.0%     35.9%

  Basic                  21.0%     8.2%       23.1%     13.3%      21.0%     12.6%

  Minimal                3.5%      1.0%       10.3%     3.9%       10.5%     4.2%

  Proficient or Above    75.6%     90.8%      66.7%     82.8%      68.5%     83.2%

  Percentage Point       -16.7%               -19.5%               -17.7%    
  Difference                                                                 
  ---------------------- --------- ---------- --------- ---------- --------- ----------

***Disability Status*.** Table 36 compares the proficiency of Reading
First students with disabilities to the statewide population. After
remaining relatively stable from 2004 to 2005, the relative percent
difference between students with disabilities in Reading First schools
and statewide worsened in 2006. The gap of -35 percent in 2004 and -36
percent in 2005 increased to -43 percent in 2006. In 2006, only 28
percent of students with disabilities in Reading First schools were
classified as proficient or above compared to 50 percent statewide.

Table 36. Comparison of Reading First Schools and the Statewide
Population:\
Proficiency Distribution by Disability Status

  ---------------------- --------- ----------- --------- ----------- --------- -----------
  **Grade 3**            **Fall                **Fall                **Fall    
                         2004                  2005                  2006      
                         WRCT**                WKCE**                WKCE**    

                         **WRF*\   **State*\   **WRF*\   **State*\   **WRF*\   **State*\
                         N* =      N* =        N* =      N* =        N* =      N* =
                         3,113**   57,168**    2,942**   56,594**    3,181**   58,187**

  **Students With                                                              
  Disabilities**                                                               

  Advanced               4.6%      15.7%       7.3%      16.4%       8.2%      17.5%

  Proficient             38.8%     50.6%       24.9%     34.0%       20.1%     32.4%

  Basic                  44.6%     27.7%       34.5%     31.5%       36.0%     31.0%

  Minimal                12.0%     6.0%        33.4%     18.2%       35.7%     19.0%

  Proficient or Above    43.4%     66.3%       32.2%     50.4%       28.3%     49.9%

  Percentage Point       -34.5%                -36.1%                -43.3%    
  Difference                                                                   

  **Nondisabled                                                                
  Students**                                                                   

  Advanced               24.9%     46.9%       28.1%     47.0%       29.7%     49.2%

  Proficient             55.9%     48.0%       44.2%     39.5%       43.4%     37.2%

  Basic                  17.3%     5.1%        21.4%     11.4%       20.2%     11.1%

  Minimal                1.9%      \<0.5%      6.3%      2.0%        6.7%      2.5%

  Proficient or Above    80.8%     94.9%       72.3%     86.5%       73.1%     86.3%

  Percentage Point       -14.9%                -16.4%                -15.3%    
  Difference                                                                   
  ---------------------- --------- ----------- --------- ----------- --------- -----------

***Economic Status*.** Table 37 compares the proficiency of economically
disadvantaged Reading First students with their statewide counterparts.
In 2006, the relative percent difference between disadvantaged students
in Reading First schools and the state average decreased slightly to -13
percent (compared with roughly -15 percent in 2004 and 2005). In Reading
First schools, 60 percent of disadvantaged students achieved proficient
or above on the state assessment whereas 69 percent attained this level
of proficiency statewide.

Table 37. Comparison of Reading First Schools and the Statewide
Population:\
Proficiency Distribution by Economic Status

  ---------------------- --------- ---------- --------- ---------- --------- ----------
  **Grade 3**            **Fall               **Fall               **Fall    
                         2004                 2005                 2006      
                         WRCT**               WKCE**               WKCE**    

                         **WRF\    **State\   **WRF\    **State\   **WRF\    **State\
                         *N* =     *N* =      *N* =     *N* =      *N* =     *N* =
                         3,113**   57,168**   2,942**   56,594**   3,181**   58,187**

  **Economically                                                             
  Disadvantaged**                                                            

  Advanced               14.6%     24.7%      14.4%     24.2%      17.6%     26.7%

  Proficient             55.8%     58.1%      44.0%     44.8%      42.2%     42.2%

  Basic                  25.5%     15.1%      28.7%     23.4%      26.6%     22.3%

  Minimal                4.1%      2.2%       12.9%     7.6%       13.6%     8.8%

  Proficient or Above    70.4%     82.8%      58.4%     69.0%      59.8%     68.9%

  Percentage Point       -15.0%               -15.4%               -13.2%    
  Difference                                                                 

  **Not Economically                                                         
  Disadvantaged**                                                            

  Advanced               31.5%     50.5%      44.7%     52.7%      46.6%     55.5%

  Proficient             49.9%     43.4%      37.5%     36.0%      36.5%     33.5%

  Basic                  16.1%     5.1%       13.2%     9.1%       13.0%     8.8%

  Minimal                2.5%      1.0%       4.6%      2.2%       4.0%      2.2%

  Proficient or Above    81.4%     93.9%      82.2%     88.7%      83.1%     89.0%

  Percentage Point       -13.3%               -7.3%                -6.6%     
  Difference                                                                 
  ---------------------- --------- ---------- --------- ---------- --------- ----------

**Comparison Group Study**

The effectiveness of Wisconsin Reading First is being studied using a
longitudinal comparison group design. In this study the comparison group
is not the state as whole, but rather a group of schools that are
similar in demographic makeup to the Reading First schools. The
Comparison Group Study addresses the following evaluation questions:

1.  Do Wisconsin Reading First students and schools demonstrate higher
    levels of reading achievement than demographically similar
    non-Wisconsin Reading First students and schools?

2.  To what extent have Wisconsin Reading First schools been effective
    in reducing the number of students reading below the basic
    proficiency level compared to similar non-Wisconsin Reading First
    schools?

3.  What factors contribute to differences in reading achievement
    between Wisconsin Reading First and non-Wisconsin Reading First
    schools?

Several analyses contributed to the evaluation of these questions. For
the first question, a three-level hierarchical linear growth model
analysis with TerraNova data was conducted to compare the growth of
students in Reading First schools to the growth of students in
comparison group schools. The second question was addressed by examining
the relative percent difference in achievement across years, again for
both groups of students (i.e., Reading First and comparison group
students). Finally, a hierarchical linear model was fit to evaluate the
third question by examining the relationship between implementation of
the Reading First program (taken from teacher survey responses) and
student achievement.

**Results**

Prior to examining the effect of Reading First status on student reading
achievement, an unconditional three-level linear growth model was fit
for the composite TerraNova score as well as the three subtest scores
(reading, vocabulary, and word analysis). The results of this
unconditional model are presented in Table 38 for the composite score
outcome (results were similar for the subtests and those results are
presented in Appendix E).

Table 38. Unconditional Three--Level Growth Model for TerraNova
Composite Score

  ------------------------------ ------------- ----------------- ----------- -------------
  **Fixed Effect**                             **Coefficient**   **se**      ***t* Ratio**

  Average final status                         631.4             4.3         147.6\*\*\*

  Average academic year learning               40.4              1.4         28.1\*\*\*
  rate                                                                       

                                                                             

  **Random Effect**              **Variance    **df**            **ChiSq**   ***p* Value**
                                 Component**                                 

  *Level 1*                                                                  

  Temporal variation             488.81506                                    

  *Level 2 (students within                                                  
  schools)*                                                                  

  Individual initial status      1182.2        2188              5726.8      0.000

  Individual academic year       69.8          2188              2139.5      \>.500
  learning rate                                                              

  *Level 3 (between schools)*                                                

  School mean status             308.4         17                620.8       0.000

  School mean learning rate      31.2          17                133.9       0.000
  ------------------------------ ------------- ----------------- ----------- -------------

\*\*\* Statistically significant at the 0.001 level.

The results in Table 38 show the average final status (631―which is the
average student score at the end of Grade 3) and average learning rate
(40 scale points per year) for the TerraNova composite score for all
students and schools in the comparison group study. Because there are no
student- or school-level predictors in an unconditional model, the
primary purpose of fitting this model was to examine the extent to which
status and growth varied both within and between schools. The model
output indicated that there was significant variation in Grade 3 scores
within schools, but not in the learning rate (p \> 0.500). The mean
Grade 3 scores varied significantly between schools, as did the learning
rate. These findings indicate that students within a school tended to
grow at similar rates as other students in their school; however, there
was considerable variation across schools in the growth rate by year.

The next step in modeling the impact of the Reading First program on
student achievement was to add level 2 (student) and level 3 (school)
predictors to the model. The analysis indicated that for the TerraNova
word analysis assessment, there was a significant impact of the Reading
First program on student test scores and growth. However, no program
impact was found for the TerraNova composite, reading, and vocabulary
assessments. Table 39 shows the final three-level model for the word
analysis assessment (the findings for the remaining three assessments
are presented in Appendix E).

Table 39. Effects of Reading First, Minority Status, Economic Status,
and School Diversity on TerraNova Word Analysis Test Performance (a
Three-Level Analysis)

  ----------------------------------------- ----------------- -------- -------------
  **Fixed Effect**                          **Coefficient**   **se**   ***t* Ratio**

  *Model for Status at End of Grade 3*                                 

  *Model for Grade 3 Status of Nonminority,                            
  Nondisadvantaged Child*                                              

  Intercept                                 643.4             2.8      226.3\*\*\*

  Reading First                             12.0              4.1      2.9\*

  Percent minority                          -27.0             6.1      -4.4\*\*\*

  *Model for Minority Gap on Grade 3                                   
  Status*                                                              

  Intercept                                 -16.6             2.4      -7.0\*\*\*

  *Model for Economic Gap on Grade 3                                   
  Status*                                                              

  Intercept                                 -10.9             1.6      -6.8\*\*\*

  Percent minority                          12.3              5.0      2.5\*

  *Model for academic year learning rate*                              

  *Model for learning rate of non-minority                             
  child*                                                               

  Intercept                                 29.8              1.4      20.9\*\*\*

  Reading First                             5.7               2.1      2.7\*

  *Model for minority gap on learning rate*                            

  Intercept                                 -2.4              1.0      -2.4\*
  ----------------------------------------- ----------------- -------- -------------

\* Statistically significant at the 0.05 level

\*\*\* Statistically significant at the 0.001 level

The most important finding shown in Table 39 is that attending a Reading
First school positively impacted word analysis test performance in Grade
3 (*t* = 2.9) and the learning rate per academic year (*t* = 2.7). These
results indicate a positive impact of Reading First implementation on
students' phonemic awareness (as modeled by the TerraNova word analysis
assessment). In addition, the output in Table 39 indicates that both a
student's ethnic minority status (minority versus nonminority) and
economic status (disadvantaged versus nondisadvantaged) impact word
analysis test performance in Grade 3. Minority students (*t* = -7.0) and
economically disadvantaged students (*t* = -6.8) tended to have lower
Grade 3 scores than nonminority and nondisadvantaged students.
Furthermore, minority students had a significantly lower growth rate
than nonminority students (*t* = -2.4).

**Comparison of WKCE Performance in WRF and Similar Non-WRF Schools**

Table 40 shows the performance of the students from the schools involved
in the Comparison Group Study across the three years. In addition to
WRCT and WKCE results, schools in the Comparison Group Study
administered both the ERDA and TerraNova assessment to their students.
(Descriptive tables of the cross-year results from these assessments are
provided in the Appendix C of this report.)

As explained earlier, it is difficult to make cross-year comparisons for
the state testing program because the assessment changed from the WRCT
to the WKCE for Grade 3 students. This change makes it difficult to make
claims about growth for any group of students. However, this change does
not prevent an examination of the performance of one group relative to
another. Table 40 shows that students in the Reading First schools did
make small gains in terms of reading proficiency relative to the
students in the comparison schools from fall 2004 to fall 2005 (going
from a relative percent difference of slightly more than -3 percent to
nearly 1 percent). However, in fall 2006, students in the Reading First
schools lagged slightly behind the students in the comparison schools
(with a relative percent difference of -7 percent).

**Table 40. Comparison of Reading First and Comparison Group Study
Schools:\
Proficiency Distribution**

  ------------------- ------------- ------------- ------------- ------------- ------------- -------------
  **Grade 3 **        **Fall 2004                 **Fall 2005                 **Fall 2006   
                      WRCT**                      WKCE**                      WKCE**        

                      **WRF*\       **Comp*\      **WRF*\       **Comp*\      **WRF*\       **Comp*\
                      N* = 406**    N* = 587**    N* = 377**    N* = 553**    N* = 436**    N* = 539**

                      **Percent**   **Percent**   **Percent**   **Percent**   **Percent**   **Percent**

  Advanced            30.0%         26.9%         30.5%         35.6%         34.4%         32.1%

  Proficient          48.3%         54.2%         44.6%         38.9%         36.9%         44.5%

  Basic               18.2%         16.4%         16.4%         17.4%         20.2%         16.0%

  Minimal             3.4%          2.6%          8.5%          8.1%          8.5%          7.4%

  Total               100.0%        100.0%        100.0%        100.0%        100.0%        100.0%

  Proficient or Above 78.3%         81.1%         75.1%         74.5%         71.3%         76.6%

  Percentage Point    -3.4%                       0.8%                        -6.9%         
  Difference                                                                                
  ------------------- ------------- ------------- ------------- ------------- ------------- -------------

**Impact of Level of Reported Implementation Factors on Student
Achievement**

A hierarchical linear model was used to evaluate the third comparison
study question by examining the relationship between implementation of
the Reading First program (taken from teacher survey responses) and
student achievement. The teacher survey was designed to measure
self-reported implementation of Reading First through sets of items
mapping to various constructs. Table 41 shows the item stem (used for
multiple items) which corresponds to each of the constructs. The Rasch
model for ordered categories (Andrich, 1978; Wright and Masters, 1982)
was used as implemented by WINSTEPS (Linacre, 2005) to develop the scale
scores for each construct.

Table 41. Teacher Survey Constructs

  ----------- --------------------------------------------------------------
  **Scale**   **Item Stem**

  q15         To what extent has Reading First professional development
              addressed each of the five essential components of reading?
              (RF Schools only)

  q16         How helpful is the Reading First professional development in
              each of the following five essential components of reading?
              (RF Schools only)

  q19         To what extent did the Reading First professional development
              you have received help you do each of the following? (RF
              Schools only)

  q20         Rate the effectiveness of the following approaches to Reading
              First professional development that you may have received in
              this school year (2005--06). (RF Schools only)

  q24         During the last month, how often did your students receive
              instruction in each of the following five essential components
              of reading? (RF Schools and Comparison Schools)

  q25         How much emphasis during the 120-minute reading block is spent
              on each of the following five essential components of reading?
              (RF Schools and Comparison Schools)

  q28         How proficient do you consider yourself in teaching each of
              the following five essential components of reading? (RF
              Schools and Comparison Schools)

  q29         How often are you using the following materials in your
              reading program this school year (2005--06)? (RF Schools and
              Comparison Schools)

  q32         Please rate the degree to which each of the following
              assessment tools impacts your reading instruction: (RF Schools
              and Comparison Schools)

  q37         Please rate the extent to which the Reading First initiative
              has helped you in each of the following areas: (RF Schools
              only)

  q38         Please rate the extent to which the Reading First initiative
              has improved your students' abilities, attitudes, or interests
              (on the average) in each of the following literacy areas: (RF
              Schools only)
  ----------- --------------------------------------------------------------

The TerraNova composite score was used as the outcome measure for this
analysis, and the HLM growth model included all students in Reading
First schools. Both the final status (at Grade 3) and learning rate (per
academic year) were modeled using each of the scales in Table 39 as
predictors. Table 42 shows the final model with the significant
predictors of student performance and growth. The most interesting
finding is that student achievement by Grade 3, on average, was higher
in schools where teachers indicated that the Reading First professional
development helped their instruction (Q19; *t* = 2.6). This result may
indicate that teachers who are committed to the program and believe in
its effectiveness were more likely to have students who showed greater
growth on the TerraNova assessment. None of the remaining 10 scales
showed a statistically significant relationship with overall assessment
performance.

**Table 42. Effects of Reading First Implementation, Minority Status,
Economic Status, and School Diversity on TerraNova Composite Score (a
Three-Level Analysis)**

  ----------------------------------------- ----------------- -------- -------------
  **Fixed Effect**                          **Coefficient**   **se**   ***t* Ratio**

  *Model for Status at End of Grade 3*                                 

  *Model for Grade 3 Status of Nonminority,                            
  Nondisadvantaged Child*                                              

  Intercept                                 652.7             1.7      378.5\*\*\*

  Q19: Extent Reading First PD helped       0.3               0.1      2.6\*
  instruction                                                          

  *Model for Minority Gap on Grade 3                                   
  Status*                                                              

  Intercept                                 -20.0             1.7      -11.9\*\*\*

  *Model for Economic Gap on Grade 3                                   
  Status*                                                              

  Intercept                                 -18.3             1.1      -17.2\*\*\*

  *Model for academic year learning rate*                              

  *Model for learning rate of nonminority                              
  child*                                                               

  Intercept                                 43.4              0.9      48.4\*\*\*

  *Model for minority gap on learning rate*                            

  Intercept                                 -6.6              0.9      -7.7\*\*\*
  ----------------------------------------- ----------------- -------- -------------

\*Statistically significant at the 0.05 level

\*\*\*Statistically significant at the 0.001 level

**Conclusions**

In the penultimate year of the program, Wisconsin Reading First shows a
number of successes with program implementation in terms of strong
structures and materials, instructional practices, teachers' views of
the program, use of assessments, professional development, and
monitoring and support. However, strong implementation of the program
often is quite similar to what occurs in non-Reading First schools. In
terms of student performance, there have been only limited gains for
students in the program.

**Structures and Materials**

Overall, staff at Reading First schools were strongly supportive of the
program, more so than their non-Reading First counterparts. For example,
89 percent of interviewees at Reading First schools reported that the
program met the needs of all students, both high- and low-achievers, in
contrast with the 54 percent of respondents from non-Reading First
schools who said the same.

**Instructional Practice**

Teachers in both Reading First and non-Reading First schools reported
relatively high levels of proficiency in instructing students in the
five essential components of reading, and both regularly instruct their
students in these areas. There were some differences between Reading
First and non-Reading First teachers' instructional practices in that:

-   Reading First teachers used small group arrangements more often than
    did non-Reading First teachers.

-   Reading First teachers were more often observed coaching students
    and engaging in higher order questions of students than their
    non-Reading First counterparts.

-   Students in Reading First classrooms tended to be more actively
    engaged than students in non-Reading First schools.

In both Reading First and non-Reading First classrooms, the observed
text materials were almost exclusively narrative.

Finally, teachers' actions suggest that they consider the program more
appropriate below third grade in that third grade Reading First teachers
spend less time on the five essential components than teachers in
kindergarten, first, or second grade.

**Perceptions of Instructional Change**

In terms of their views of practice within their schools, teachers were
quite supportive of the Reading First program. They credited the program
with:

-   Increasing their knowledge about effective reading instruction

-   Increasing collaboration among teachers

-   Increasing consistency of instruction within their schools

-   Providing them with coordinators who offer meaningful feedback on
    their classroom practice

**Use of Assessments**

Reading First and non-Reading First teachers both use assessment data
primarily to plan instruction, group students, and determine
interventions.

**Professional Development**

Most Reading First teachers, principals, and coordinators found the
professional development useful and beneficial, providing valued
information on the five essential components and best practices, and
differentiating instruction to meet student needs. They reported
favorably on the support provided by INSIGHT, the external professional
development provider. Just more than one half (55 percent) of Reading
First teachers reported that that their instruction had been observed by
the INSIGHT representative, and all of those who had been observed said
that the feedback they had received after the observation had been
useful. They credited the value of this process partially to the fact
that the INSIGHT representative was someone with whom they had an
ongoing, respected relationship.

Compared to non-Reading First professional development for teachers,
Reading First professional development was more likely to cover the five
essential components, differentiated instruction, classroom management,
and the use of literacy centers. Non-Reading First teachers were more
likely to receive professional development in writing and assessments.

**Monitoring and Support**

Both Reading First and non-Reading First schools reported strong
district support of their literacy programs, indicating that this
support took the form of:

-   Professional development

-   Open lines of communication

Reading First schools were more likely to receive support in the form
of:

-   Individual rather than group support from their literacy specialist
    (the coordinator)

-   Classroom observations by their coordinators and principals

In terms of future support for the program after funding has ended,
staff at Reading First schools said that the model is sustainable and
that it was most important to retain the focus on the five essential
components of reading. They were most concerned about losing funding for
professional development, coordinators, and materials. They also
expressed concern about how to replace staff who had developed expertise
in the program; the likelihood of losing staff who have developed
expertise was the barrier to sustaining the Reading First program cited
most frequently by Reading First staff.

**Reading First Schools Versus Statewide WKCE Performance**

Comparisons of Reading First students' performance and the performance
of students across the state show little change since the advent of the
program, although there was a decrease in the achievement gap for
African-American students from 2004 to 2005 that appears to have
solidified in 2006. In addition, there was a slight increase in the gap
for students with disabilities at Reading First schools compared with
schools across the state.

**Reading First Schools Versus Comparison Group Schools**

Looking at the performance of Reading First students in relation to the
performance of students in comparable schools chosen for the comparison
study, there were small differences. Reading First schools showed
greater achievement at Grade 3 and higher growth rates than their
counterparts in the comparison group on the TerraNova word analysis
assessment (an indicator of phonemic awareness). However, no program
impact was found for the TerraNova composite, reading, and vocabulary
assessments. Furthermore, although Reading First schools made small
gains in reading proficiency from 2004 to 2005, these schools lagged
slightly behind schools in the comparison group in 2006 (in terms of
percentage of students at or above proficiency on the WKCE).

> **References**

Andrich, D. (1978). A rating formulation for ordered response
categories. *Psychometrika, 43*(4), 561--573.

Dey, I. (1993). *Qualitative data analysis: A user-friendly guide for
social scientists*. New York: Routledge.

Embretson, S. E. (1996). Item response theory models and spurious
interaction effects in factorial ANOVA designs. *Applied Psychological
Measurement, 20*(3), 201--212.

Embretson, S. E. (2006). The continued search for nonarbitrary metrics
in psychology. *American Psychologist, 61*(1), 50--55.

LeCompte, M. D. (2000). Analyzing qualitative data. *Theory Into
Practice, 39*(3), 146--154.

Linacre, J. M. (2004). Rasch model estimation: Further topics. In E. V.
Smith Jr. & R. M. Smith (Eds.), *Introduction to Rasch measurement:
Theory, models and applications* (pp. 48--72). Maple Grove, MN: JAM
Press.

Linacre, J. M. (2005). *WINSTEPS Rasch measurement computer program*.
Chicago: Winsteps.com.

Maxwell, S. E., & Delaney, H. D. (1985). Measurement and statistics: An
examination of construct validity. *Psychological Bulletin, 97*(1),
85--93.

Meyer, R., Camburn, C., Kimball, S., Milanowski, T., Morgan, E.,
Lowenhaupt, R., et al. (2006). *Study of literacy coach initiative:,
Phase 1 report*. Madison: Wisconsin Center for Education Research.

Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are
teacher effects?

> *Educational Evaluation and Policy Analysis, 26*(3), 237--257.

Perline, R., Wright, B. D., & Wainer, H. (1979). The Rasch model as
additive conjoint measurement. *Applied Psychological Measurement,
3*(2), 237--255.

Rasch, G. (1960). *Probabilistic models for some intelligence and
attainment tests.* Copenhagen: Danish Institute for Educational
Research. (Exp. Ed., 1980. Chicago: University of Chicago Press).

Schumacker, R. E. (2004). Rasch measurement: The dichotomous model. In
E. V. Smith Jr. & R. M. Smith (Eds.), *Introduction to Rasch
Measurement: Theory, models and applications* (pp. 226--257). Maple
Grove, MN: JAM Press.

State of Wisconsin. (2003, May). *Reading First grant proposal Title 1,
Part B, Subpart 1*. Proposal to U.S. Department of Education.

Taylor, B. M., & Pearson, P. D. (2000). *The CIERA school change
classroom observation scheme*. Minneapolis, MN: University of Minnesota.

Taylor, B. M., Pearson, P. D., Peterson, D. S., & Rodriguez, M. C.
(2002). *The CIERA school change project: Supporting schools as they
implement home-grown reading reform.* Ann Arbor, MI: Center for the
Improvement of Early Reading Achievement. Retrieved October 10, 2007,
from http://www.ciera.org/library/reports/inquiry-2/2-016/2-016a.pdf

Taylor, B. M., Pearson, P. D., Peterson, D. S., & Rodriguez, M. C.
(2003). Reading growth in high-poverty classrooms: The influence of
teacher practices that encourage cognitive engagement in literacy
learning. *Elementary School Journal, 104*(1), 3--28.

Wolfe, E. W. (2004). Equating and item banking with the Rasch model. In
E. V. Smith Jr. & R. M. Smith (Eds.), *Introduction to Rasch
measurement: Theory, models and applications* (pp. 366--390). Maple
Grove, MN: JAM Press.

Wright, B. D. (1977). Solving measurement problems with the Rasch model.
*Journal of Educational Measurement, 14*(2), 97--116.

Wright, B. D., & Linacre, J. M. (1989). Observations are always ordinal;
measurements, however, must be interval. *Archives of Physical Medicine
and Rehabilitation, 70*(12),\
857--960.

Wright, B. D., & Masters, G. N. (1982). Rating scale analysis: Rasch
measurement. Chicago: MESA Press.

Wright, B. D., & Stone, M. H. (1979). *Best test design*. Chicago: MESA
Press.

#