+--------------------------+-------------------------------------------+
| ![                       | Reading First in Tennessee                |
| ](media/image1.jpeg){wid |                                           |
| th="2.123611111111111in" | Research Study Report                     |
| height                   |                                           |
| ="0.9805555555555555in"} | 2006 -- 2007                              |
|                          |                                           |
| Center for Research in   |                                           |
| Educational Policy       |                                           |
|                          |                                           |
| The University of        |                                           |
| Memphis                  |                                           |
|                          |                                           |
| 325 Browning Hall        |                                           |
|                          |                                           |
| Memphis, Tennessee 38152 |                                           |
|                          |                                           |
| Toll Free:               |                                           |
| 1-866-670-6147           |                                           |
+--------------------------+-------------------------------------------+

![](media/image2.jpeg){width="7.875in" height="5.825in"}

+--------------------------+-------------------------------------------+
| ![                       | Reading First in Tennessee                |
| ](media/image1.jpeg){wid |                                           |
| th="2.123611111111111in" | Research Study Report                     |
| height                   |                                           |
| ="0.9805555555555555in"} | 2006-2007                                 |
|                          |                                           |
| Center for Research in   |                                           |
| Educational Policy       |                                           |
|                          |                                           |
| The University of        |                                           |
| Memphis                  |                                           |
|                          |                                           |
| 325 Browning Hall        |                                           |
|                          |                                           |
| Memphis, Tennessee 38152 |                                           |
|                          |                                           |
| Toll Free:               |                                           |
| 1-866-670-6147           |                                           |
+--------------------------+-------------------------------------------+
|                          |                                           |
+--------------------------+-------------------------------------------+
|                          | November 2007                             |
+--------------------------+-------------------------------------------+
|                          |                                           |
+--------------------------+-------------------------------------------+
|                          | Anna Grehan                               |
|                          |                                           |
|                          | Lynn Harrison                             |
|                          |                                           |
|                          | Guler Boyraz                              |
|                          |                                           |
|                          | Ying Huang                                |
|                          |                                           |
|                          | **Center for Research in Educational      |
|                          | Policy**                                  |
+--------------------------+-------------------------------------------+

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

**EXECUTIVE SUMMARY**

**INTRODUCTION**

Reading First is a U.S. Department of Education program that provides
funds to state agencies for distribution to districts and/or schools to
improve K-3 classroom reading instruction. As the academic cornerstone
of No Child Left Behind (NCLB), Reading First recognizes the critical
need to improve student reading skills in grades K-3. Principles of
Reading First are based on recommendations from the National Reading
Panel, with the overall purpose to "enable all students to become
successful early readers" (Reading First, 2003). It is widely observed
that children experiencing reading difficulties tend to cluster in
low-performing, high need schools where they lag behind their peers in
more affluent schools. Funds provided through Reading First are intended
to support early reading by eliminating reading deficits through high
quality, research-based instruction for K-3 students. Reading First
grants also allow schools to select, implement and provide professional
development for teachers using scientifically based instructional
reading programs. In addition, states must ensure accountability through
on-going, valid, reliable screenings of students for reading
difficulties, as well as diagnostic and classroom based assessments.
Students must show progress before the initial grants awarded under
Reading First are funded in additional years. Thus, Reading First, along
with the programs authorized under Title I, focus on improving student
achievement for all students, especially children in the nation's most
disadvantaged schools and communities.

The Center for Research in Educational Policy (CREP) at the University
of Memphis has been engaged by the Tennessee State Department of
Education to conduct an evaluation of the state's Reading First
programs. The evaluation plan for the Reading First grant includes: (1)
a formative evaluation of the schools receiving Reading First funds; and
(2) a research study to compare Reading First schools to matched control
schools, with analysis that includes classroom observations of
instructional practices, program-element survey responses by teachers, a
review and summary of Reading First schools' implementation benchmarking
goals, and assessments of student reading skills. Both the formative and
research evaluation components complement one another and are designed
to provide a rich and full picture of intended and achieved Reading
First goals and objectives.

**RESEARCH QUESTIONS**

The purpose of this research study is to assess whether the Reading
First program has improved student performance in reading. The research
study is longitudinal in nature, with the 2006-2007 school year the
first year of data collection. This report reflects assessments,
observations, and questionnaires for year one only; subsequent reports
will analyze the progress made over time. The research study is
structured to address the following research questions:

1.  Do kindergarten through 3^rd^ grade students at Reading First
    schools have significantly higher reading achievement scores than
    students at control schools once socioeconomic status and pre-test
    scores are controlled?

2.  Are Reading First program effects on reading achievement sustained
    over time?

3.  Which Reading First program characteristics are associated with the
    greatest increases in reading achievement?

4.  Are classroom practices significantly different in Reading First
    schools compared to control schools?

5.  Do schools with fully implemented Reading First programs have
    greater gains in reading achievement than those with lower levels of
    implementation?

**DESIGN**

The first year of the research evaluation period extended from September
2006 through June 2007. The research design employed in the evaluation
study is a multi-site replication design (Slavin & Madden, 1993). Using
this approach, each Program (Reading First)-Control school comparison is
considered a replication of the other sites involved in the study. For
this study, there were 12 matched pairs, for a total of 24 schools. The
study is longitudinal in nature, allowing comparisons to be made over
time, and providing an assessment of trends, fidelity to the model, and
the sustainability of results. The design is based on both quantitative
and qualitative data collected from classroom observations, teacher
questionnaires, and school-developed Reading First program
implementation benchmarks. Individual student testing was conducted
during fall of 2006 and spring 2007; results have been incorporated in
this report.

**Sample and Participants**

In the summer of 2006, twelve Reading First schools were randomly
selected as the treatment schools. These schools are located both in
urban and rural areas and are representative of Reading First programs
across the state. Additionally, twelve control schools were selected
that are demographically similar to the treatment schools. These 24
schools will be the sites for the 3-year research study. Kindergarten
students were initially selected for participation in year one of the
study. All kindergarten students from the Reading First and control
sites were invited to participate in the study and parental permission
was required. During year one, there were 1,716 kindergarten students in
the study, with 861 in the treatment group and 855 in the control group.
These students will be followed for the duration of the study, thus
providing a longitudinal perspective of the effects of Reading First
students as they progress through school over multiple years.

**INSTRUMENTATION**

**Classroom Observation Measures**

**Literacy Observation Tool (LOT©)**

The LOT was developed by researchers at the Center for Research in
Educational Policy, College of Education, The University of Memphis
(Smith, Ross, & Grehan, 2002). The LOT is an instrument for observing
teachers' instructional practices in K-3 classrooms where teachers are
engaged in teaching reading and using other reading materials (e.g.,
reading centers, manipulatives, etc.). The LOT was designed to assist
schools in evaluating the effectiveness of teacher implementation of
research-based reading strategies and practices.

**Teacher Surveys**

**Reading First Teacher Questionnaire (RFTQ©)**

All K-3 teachers and special education teachers who taught reading were
asked to complete the RFTQ, which contains 27 items to which teachers
responded using a five-point Likert-type scale that ranges from (1)
Strongly Disagree to (5) Strongly Agree. Items assessed included the
specific program elements of Reading First such as: general impressions
of professional development, support provided by the Literacy Leader and
the Reading Cadre, teacher support for the program, impacts on student
achievement, changes in teaching and assessment, and understanding of
the Reading First program.

**Reading First Teacher Questionnaire-Control (RFTQ-C)**

A similar questionnaire (RFTQ-C) was developed for use in the comparison
schools. As with the RFTQ, all K-3 teachers and special education
teachers who taught reading were asked to complete the RFTQ-C, which
contained 27 items to which teachers responded using a five-point
Likert-type scale that ranged from (1) Strongly Disagree to (5) Strongly
Agree. Because comparison schools did not receive Reading First funds,
all references to Reading First were replaced with phrases such as "our
school's literacy program" in the RFTQ-C.

**Implementation Benchmarking Tool**

The Implementation Benchmarking Tool was developed by CREP and provides
generic benchmarks which schools may then use to assist them in refining
and developing goals directly geared to their school's implementation of
the Reading First grant. The five components of effective reading
instruction, Reading First assessments, resources, and organization are
included in the generic model. For the purposes of this study, only
treatment schools used the program benchmarks and differences between
treatment schools were examined.

**Literacy Tests**

Measures were derived from the following sources:

*Peabody Picture Vocabulary Test (PPVT)*. The PPVT-IV (Dunn & Dunn,
2006) is an individually administered norm-referenced assessment of
receptive vocabulary that measures listening comprehension of spoken
words for children. All children in the study were tested in the fall of
2006 with the PPVT to establish initial literacy skills, and again in
spring of 2007 to compare growth throughout the year.

*Dynamic Indicators of Basic Early Literacy Skills (DIBELS)*. DIBELS is
a tool for early identification of children with potential literacy
problems and an assessment of response to instruction. The DIBELS
assessment is designed to enable educators to modify their approach if a
student is not on course to achieve reading goals. The Early Childhood
Research Institute on Measuring Growth and Development at the University
of Oregon constructed DIBELS (2000). The Institute reports it has
validated the instrument's ability to predict outcomes, and has tested
its reliability with young children across the country. In this study,
subtests of DIBELS that were given include: letter naming fluency,
phoneme segmentation fluency, and nonsense word fluency.

**PROCEDURE**

In 2006, 11 site observers were selected to conduct observations for the
research study (both treatment and control schools). Site researchers
conducted a total of five observations at each school throughout the
fall and spring semesters. Generally, observations were scheduled a
month apart and distributed throughout the days of the week, for a
complete representation of K-3 reading instruction. Teacher
questionnaires were also administered to teachers during the spring of
2007.

Test administrators were selected and trained to conduct individual
student testing in the fall of 2006 and the spring of 2007. The PPVT-4
was administered in the fall and then again in the spring. Also in the
spring, subtests of DIBELS were administered. Only kindergarten students
were tested in the treatment and control schools.

**RESULTS**

**Classroom Observation Results**

**Literacy Observation Tool (LOT©)**

***Descriptive Results***

**Instructional Orientation**

For both the treatment and control classrooms, the primary instructional
orientation utilized was direct instruction to the whole class, observed
frequently or extensively in 63.4% of the treatment classrooms and 88.1%
of the control classrooms. However, teachers in treatment classrooms
were observed employing a wider variety of instructional orientations
than their control counterparts including learning centers (20.0% for
treatment and 3.4% for control) and small group instruction (16.7% for
treatment and 3.4% for control). Cooperative or collaborative learning
was either rarely or not observed in the majority of both treatment and
control classrooms.

### Instructional Components

LOT results also suggested treatment teachers were more likely to employ
a wider array of instructional techniques in their classrooms than
teachers in the control classrooms. The greatest differences were seen
in the various elements of text comprehension instruction. Observations
reflected greater use of all elements of text comprehension by treatment
teachers; most noteworthy, treatment teachers were observed utilizing
explicit comprehension instruction frequently or extensively in 33.3% of
the classrooms; for control teachers, this strategy was observed
frequently or extensively in 8.5% of the classrooms. In addition,
treatment teachers were frequently or extensively observed engaged in
explicit vocabulary instruction in 20% of the observations---control
teachers were not observed frequently or extensively utilizing this
practice. Reading First teachers were also observed utilizing fluency
activities more often than control teachers, especially with an emphasis
on students reading together orally, which was frequently or extensively
observed in 43.3% of the treatment classrooms, compared with 25.4% of
the control classrooms.

### Student Activities

The majority of observations reflected students in both treatment and
control schools were rarely or never engaged in the following: reading
self-selected materials; writing independently in response to reading;
or engaging in formal testing.

### Learning Environment

The observation data suggested positive learning environments in both
treatment and control classrooms. The majority of classrooms were
conducive to cooperative interactions and were effectively managed with
teachers actively monitoring and students highly engaged.

### Visible Print Environment

A print rich environment was observed in both treatment and control
classrooms. The majority of all classrooms had classroom libraries, word
walls, and a displayed alphabet readily available to students. Evidence
of students' writing or their work products was less prevalent in both
treatment and control and classrooms. In treatment classrooms, 71.6% of
the classrooms frequently or extensively had student work displayed;
while for control classrooms, 61.0% of the classrooms exhibited
students' work.

**Materials Used**

LOT results indicated that treatment classrooms had a wider variety of
frequently and extensively used materials than control classrooms.
Materials most often observed in use in treatment classrooms were: chart
tablets/white boards (36.6%), word and vocabulary materials (33.4%),
basal texts (31.6%), worksheets or workbooks (21.7%), fiction books
(13.3%), and computers (11.7%). In control classrooms, the materials
most frequently observed in use were: basal texts (22.0%), chart
tablets/white boards (22.0%), fiction books (15.3%), word/vocabulary
materials (15.3%), and worksheets or workbooks (6.8%).

***Inferential Results***

There are 41 items on the LOT, all of which are measured on the same
5-point Likert-scale: 0=Not observed, 1=Rarely, 2=Occasionally,
3=Frequently and 4=Extensively. As a result, the alpha level was set to
a Bonferroni adjusted .0012 (.05 divided by 41 questions). The number of
returned LOT observations was almost identical for the treatment group
(N=60) and the control group (N=59). The Wilcoxon Mann Whitney was
employed for the LOT observations.

Only 3 out of 41 items were found to be statistically significant
between the two groups. "Small group" and "Learning centers" in the
Instructional Orientation section had a higher mean and mean rank score
for treatment schools and were observed significantly more frequently;
while "Whole class" (also in the Instructional Orientation section) had
a higher mean and mean rank score and was observed significantly more
frequently for the control group. Although not statistically
significant, the majority of the remaining 38 LOT items exhibited a
higher mean and mean rank score in the treatment schools relative to the
control schools, revealing a higher frequency in the treatment schools.

**Survey Results**

**Reading First Teacher Questionnaire (RFTQ) and (RFTQ-C)**

***Descriptive Results***

Although the majority of responses were positive for both groups, in
general, teachers in treatment schools were more positive than the
control schools teachers. Of the 27 items listed on the questionnaire,
over 80% of the Reading First teachers agreed with 21 of them; for
control teachers, there were 10 items in which over 80% of the teachers
were in agreement. The most striking differences were seen in the areas
of: the impact the literacy program had on instruction (95% of the
treatment teachers responded positively versus 61.4% of the control
teachers); support from the Literacy Leader (92.7% of the treatment
teachers positively responding versus 72.0% of the control teachers),
the amount of feedback and coaching received (86.6% positive in the
treatment group, 58.2% in the control group) and the availability of
scientifically-based materials (97.7% for the treatment teachers and
77.2% for the control teachers). Items that were less positive were
consistent among the two groups and included: planning time, parental
involvement, and student misbehavior.

***Inferential Results***

An inferential analysis was conducted based on the available
questionnaires returned: 262 from the treatment schools and 189 from the
control schools. A total of 27 questions compose the RFTQ, and each is
measured on a 5-point Likert Scale: 1=Strongly Disagree, 2=Disagree,
3=Neutral, 4=Agree, and 5=Strongly Agree. A Shapiro-Wilk test for each
of the 27 questions was statistically significant at an alpha level of
.0018 (.05 divided by 27 questions), revealing that the data were not
normally distributed. Because the assumption of normality was not
satisfied for the data, non-parametric analysis, Wilcoxon Mann Whitney,
was employed to draw the statistical inference. The alpha level was
adjusted to 0.0018, taking 27 questions into account.

All of the RFTQ questions were rated more positively (mostly Agree or
Strongly Agree) in the treatment group as opposed to the control group.
The mean scores in the treatment group were all at least 4.00 (on a 5
point scale) with the exception of three questions, while the control
group had over half of questions (15 of 27) with a mean below 4.00.
There was a statistically significant difference between groups in favor
of the treatment group on all but three questions, an indication of
highly positive responses from the treatment teachers.

**Implementation Benchmarking Tool**

***Descriptive Results***

During the benchmark process, a team from each of the Reading First
schools developed goals with measurable criteria and then classified the
school's progress towards attaining the goals. Goals fall into the
categories of: Curriculum, Instruction, and Organization. Each area is
evaluated by the team and deemed to be in: Phase 1 (the lowest level of
implementation); Phase 2 (some level of attainment, yet not fully
realized) or Phase 3 (fully attained goal). There were 10 Round I
schools; therefore, during the 2006-2007 school year, these schools were
in their fourth year of implementation of Reading First. The remaining
were Round 2 schools and were in the third year of implementation.
Therefore, most of the schools had experienced and fully trained
teachers who were fully implementing Reading First and considered
themselves to fall in the Phase 3 level of attainment.

For the 12 schools in the research study, 11 reported to be in Phase 3
for the *Curriculum* benchmark goal, indicating that
scientifically-based reading program components are cross-referenced to
state and local learning standards and monitored regularly for
effectiveness in increasing student achievement. Additionally, a Phase 3
level also indicates that any modifications in curriculum are made to
accommodate the requirements of Reading First. Results from the RFTQ
supported the benchmark rankings. Almost all of the respondents (98.9%)
of the Reading First teachers agreed that all instructional elements
were based on scientifically-based reading research; in addition; 96.2%
agreed that elements of Reading First were effectively integrated to
help meet school improvement goals. Over 80 percent (83.2%) responded
positively that, because of Reading First, teachers spent more time
together developing curriculum and planning instruction.

Regarding the overall *Instruction* benchmark goal, 9 of the 12
treatment schools reported they were in Phase 3, which indicates that
all teachers routinely and skillfully use research-based teaching
strategies for the entire reading instruction time. It also indicates
that components of the reading program are taught in a coherent and
systematic fashion. The remaining three treatment schools reported being
in Phase 2 for the *Instruction* goal, which suggests most teachers use
a variety of research-based teaching strategies to teach reading. This
would also indicate that systems are in place to monitor/evaluate
students' skill levels and provide appropriate coaching and feedback to
students. The majority of teachers from the treatment schools (97.7%)
agreed that because of Reading First, systematic and explicit
instruction in the five essential elements of reading (phonemic
awareness, phonics, vocabulary, fluency, and comprehension) occurred
daily in their classrooms.

The review of LOT data provides a fuller context for interpreting
benchmarking goals. Although Reading First teachers utilized a wider
variety of instructional orientations and techniques compared with their
control counterparts, more evidence of Phase 3 attainment would be
expected for schools in their third and fourth years of implementation.
The majority of Reading First teachers were either rarely or not
observed engaged in phonemic awareness or phonics instruction, yet
one-third of the schools (4 schools) indicated a Phase 2 for this area.
This suggests perhaps the benchmarking team overestimated instructional
focus in this area. However, in the area of text comprehension, 9 of 12
schools reported a Phase 2 attainment and LOT data supported this
finding with the majority of Reading First teachers either never or
rarely observed utilizing text comprehension techniques.

For the *Organization* benchmark goal, 11 treatment schools reported
that they were in Phase 3 indicating that all major organizational
structures that support the school goals and the Reading First program
are in place and functioning appropriately and effectively. The area of
greatest concern appeared to be parental involvement. Six of the schools
reported a Phase 2 attainment in this area, and this is further
reflected in the teacher questionnaire.

**Literacy Tests Results**

Kindergarten students in the 24 schools were assessed with the PPVT in
the fall and spring and with subtests of DIBELS in the spring only. A
rigorous Hierarchical Linear Model (HLM) was utilized to gauge academic
achievement over the course of the school year and to assess the Reading
First program's effect on achievement in general. A total of 1,427
kindergarten students with fall and spring scores were included in the
study. A more in-depth analysis of the statistical procedures and
results are provided in the body of the report; however, the following
are noteworthy findings:

-   All students, regardless of group (either treatment or control),
    posted substantial gains in PPVT scores over the course of the
    2006-2007 school year.

-   Reading First students who were lower performers on the fall PPVT
    assessments had higher posttest scores than their control
    counterparts.

-   For the PPVT, non-African American students outperformed African
    American students.

-   On the PPVT, female students outpaced male students.

-   Overall, Reading First students scored lower on the fall PPVT
    compared to those in the control schools; yet, their scores were
    relatively higher in the spring.

-   Using the PPVT as a covariate to predict scores, Reading First
    students who scored below average on the fall PPVT assessments,
    scored statistically significantly higher on each of the three
    DIBELS subtests than the control students who scored below average
    on the fall PPVT assessments.

**CONCLUSIONS**

**Do kindergarten through 3^rd^ grade students at Reading First schools
have significantly higher reading achievement scores than students at
control schools once socioeconomic status and pre-test scores are
controlled?**

The initial achievement results for kindergarten students in Reading
First schools were positive. Although all students, regardless of group
status, posted substantial gains in PPVT scores of the 2006-2007 school
year, those in the Reading First classrooms who were lower performers
during the fall assessment had higher spring PPVT scores than those in
the control group who were low performers. Additionally, Reading First
students who scored below average on the fall PPVT assessment scored
statistically significantly higher on each of the three DIBELS subtests
than the control students who scored below average on the fall PPVT
assessment. These results are promising and indicate that the
instructional techniques applied in the kindergarten classrooms are
helping low-performing students acquire the necessary pre-reading and
beginning reading skills to be successful in first grade.

**Are Reading First program effects on reading achievement sustained
over time?**

Although initial results are positive concerning reading achievement,
years 2 and years 3 of the research study will be useful in determining
the sustainability of the impact of Reading First instruction.

**Which Reading First program characteristics are associated with the
greatest increases in reading achievement?**

In year one of the study, in general, there were no substantial
differences between Reading First schools in terms of implementation,
literacy instruction, and teacher perceptions. Likewise, student
achievement was consistent, and positive, among the 12 Reading First
schools. Therefore for the baseline year, no strong conclusions can be
made regarding specific program characteristics as determinants for
reading achievement. The overall positive results on the DIBELS
subtests, especially for those children with initially low PPVT scores,
suggests that instruction that emphasized basic letter recognition and
phonemes had an impact on student growth and progress.

**Are classroom practices significantly different in Reading First
schools compared to control schools?**

The data collected from the LOT indicates that instructional practices
differed between Reading First classrooms and control classrooms. In
general teachers in Reading First classrooms utilized a wider variety of
instructional orientations than control teachers, including a greater
use of both learning centers and small group instruction. Reading First
teachers were also more likely to utilize a more varied array of
techniques in fluency instruction, vocabulary, and text comprehension.
The LOT data also suggested that all classrooms had very positive
learning environments with students actively engaged and effective
classroom management practices in use. In general, print rich
environments were also evident in all classrooms; however, the Reading
First classrooms were more likely to have evidence of student writing on
display.

Results from the teacher questionnaires also suggested that classroom
practices differed between the treatment and control groups. For the
Reading First teachers, 95.0% agreed that Reading First had an impact on
instruction; while for control teachers, 61.4% agreed that their
literacy program had an impact on instruction. Reading First teachers
were also more positive concerning the availability of scientifically-
based materials (97.7% positively responding) than their control
counterparts (77.2% positively responding).

**Do schools with fully implemented Reading First programs have greater
gains in reading achievement than those with lower levels of
implementation?**

All of the schools in the research study have multiple years of
experience in implementing the Reading First program. Implementation
benchmarks suggested that the majority of both Round 1 and Round 2
schools considered themselves to be in Phase 3 implementation, or highly
implementing all aspects of the Reading First program. Therefore there
would appear to be few discernable differences in implementation of
Reading First among the 12 schools chosen for the study. With the
initial positive achievement scores across all Reading First research
schools, full implementation appears to have an impact on student
progress. However, without the ability to compare to lower-implementing
schools, stronger conclusions cannot be drawn.

**TENNESSEE READING FIRST**

**2007 RESEARCH EVALUATION REPORT**

Reading First (RF) is a U.S. Department of Education program that
provides funds to state agencies for distribution to districts and/or
schools to improve K-3 classroom reading instruction. Reading First, the
academic cornerstone of the No Child Left Behind Act (NCLB), recognizes
the critical need to improve student reading skills in grades K-3, based
on recommendations from the National Reading Panel. The purpose of
Reading First is to "enable all students to become successful early
readers" (Reading First, 2003). Data indicate that children experiencing
reading difficulties usually cluster in low-performing, high need
schools where they lag behind their peers in more affluent schools.
Reading First is intended to support early reading by eliminating
reading deficits through high quality, research-based instruction for
K-3 students. Schools receiving Reading First funds are expected to
utilize scientifically based instructional programs as well as select,
implement and provide professional development for teachers. Reading
First also requires that states ensure accountability through on-going,
valid, reliable screening for reading difficulties and diagnostic and
classroom based assessments. Schools must demonstrate their students'
progress before initial grants are extended for additional years. Thus,
Reading First, along with the programs authorized under Title I, focuses
on improving student achievement for all students, especially children
in the nation's most disadvantaged schools and communities.

Since 2003, all 50 states have approved Reading First plans in place and
have received awards. The 2006 Federal budget allocated \$1.1 billion
for Reading First. States are responsible for the quality and
evidence-based content of the local programs they fund. States are
further required to make certain that professional development to
improve K-3 teachers' instructional skills in reading is based on proven
scientific findings. States competitively award sub-grants to eligible
(as defined in states' requests for applications) Local Education
Agencies (LEAs).

Annually, Tennessee has received nearly 20 million dollars in RF funds
since 2003, and is projected to receive approximately 116 million
dollars by 2008. In Tennessee, LEAs are required to demonstrate that,
among all applicants for Reading First funds, the schools with the
highest numbers or percentages of students reading below grade level in
K-3, are selected. In addition, the LEA must have jurisdiction over at
least one of the following:

-   A geographic area that includes an area designated as an Empowerment
    Zone or an Enterprise Community, as defined by part 1 of subchapter
    U of Chapter 1 of the Internal Revenue Code; or

-   A significant number or percentage of schools that are identified
    for school improvement under Title I, Part A; or

-   The highest numbers or highest percentages of children who are
    counted for allocations under Title I, Part A, in comparison to
    other LEAs in the State.

To be eligible for RF funds, Tennessee designated districts with schools
that had the lowest twenty percent (20%) NCE scores in reading on the
third (3rd) grade TCAP assessment and/or schools identified for school
improvement under Title I, Part A and/or schools with the highest
percentage or numbers of children counted for allocations under section
Title I, Part A. The number of LEAs eligible for Reading First
sub-grants in 2003 was 40, or 29% of the total 138 LEAs. In January
2004, Tennessee awarded 16 LEAs and 56 schools sub-grants of
approximately \$200,000 for three years (referred to as Round I
schools). In 2005, an additional 19 schools (Round II) received funding
bringing the total number of schools participating in the program to 75.
Tennessee plans to have two three-year grant cycles during the six-year
term of the Reading First program. LEAs that have shown strong reading
gains each year and have significantly increased student achievement by
the end of the initial grant period will be eligible to participate in
the second three-year grant term.

The Center for Research in Educational Policy (CREP) at The University
of Memphis was engaged by the Tennessee State Department of Education to
conduct an evaluation of the state's Reading First programs. The
evaluation plan for the Reading First Grant included: (1) a formative
evaluation of the schools receiving Reading First funds; and (2) a
longitudinal research study comparing Reading First schools to matched
control schools with a summative report on classroom observations of
instructional practices, program-element survey responses by teachers,
and assessments of student reading skills. Both the formative and
summative evaluation components complement one another and are designed
to provide a rich and full picture of intended and achieved Reading
First goals and objectives.

The research study was designed to provide a tightly-controlled and
focused examination of Reading First program impacts by using a
quasi-experimental matched "treatment-control" school design with a
representative sample of Tennessee schools. Additionally, the research
study is longitudinal in nature, allowing comparisons to be made over
time, and providing an assessment of the sustainability of results. The
2006-2007 school year was the inaugural year, or year one, of the study
and focused on kindergarten students in the Reading First and control
schools. Year two will follow these same students in the first grade
year, and year three will examine the original student population in
their third grade year. This report contains results from year one only
and includes: (a) comparisons of results from individually-administered
reading tests, (b) data collected from trained researchers who observed
and assessed classroom instruction, (c) teacher perceptions and
understanding of literacy components and implementation, and (d)
implementation benchmarking analysis.

**Research Questions**

The Reading First research study evaluation was structured to answer the
following questions (summarized in Table 1):

1.  Do K-3 students at Reading First schools have significantly higher
    reading achievement scores than students at matched control schools,
    once socioeconomic status and pre-test scores are controlled?

2.  Are Reading First program effects on reading achievement sustained
    over time?

3.  Which Reading First program characteristics are associated with the
    greatest increases in reading achievement?

4.  Are classroom practices significantly different in Reading First
    treatment schools compared to control schools?

5.  Do schools with fully implemented Reading First programs have
    greater gains in reading achievement than those with lower levels of
    implementation?

**Table 1: Research Question Summary**

+-----------------------+-----------------------+----------------------+
| **Evaluation          | **Participants**      | **Data Sources**     |
| Question**            |                       |                      |
+-----------------------+-----------------------+----------------------+
| **Do Kindergarten     | -   In first year of  | -   PPVT-IV (fall    |
| through 3^rd^ grade   |     study, all        |     2006, spring     |
| students at Reading   |     kindergarten      |     2007; spring     |
| First schools have    |     students at 24    |     2008; spring     |
| significantly higher  |     elementary        |     2009)            |
| reading achievement   |     schools; this     |                      |
| scores than students  |     cohort of         | -   DIBELS (spring   |
| at matched control    |     students will be  |     2007, spring     |
| schools once          |     followed as first |     2008, spring     |
| socioeconomic status  |     and second grade  |     2009)            |
| and pre-test scores   |     students in years |                      |
| are controlled?**     |     2 and 3 of the    | -   GORT-4 (spring   |
|                       |     study             |     2008, spring     |
|                       |                       |     2009)            |
|                       |                       |                      |
|                       |                       | -   TCAP scores      |
+-----------------------+-----------------------+----------------------+
| **Are program effects | -   In first year of  | -   PPVT-IV (fall    |
| on reading            |     study, all        |     2006, spring     |
| achievement sustained |     kindergarten      |     2007; spring     |
| over time?**          |     students at 24    |     2008; spring     |
|                       |     elementary        |     2009)            |
|                       |     schools; this     |                      |
|                       |     cohort of         | -   DIBELS (spring   |
|                       |     students will be  |     2007, spring     |
|                       |     followed as first |     2008, spring     |
|                       |     and second grade  |     2009)            |
|                       |     students in years |                      |
|                       |     2 and 3 of the    | -   GORT-4 (spring   |
|                       |     study             |     2008, spring     |
|                       |                       |     2009)            |
|                       |                       |                      |
|                       |                       | -   TCAP scores      |
+-----------------------+-----------------------+----------------------+
| **Which Reading First | -   In first year of  | -   PPVT-IV (fall    |
| program               |     study, all        |     2006, spring     |
| characteristics are   |     kindergarten      |     2007; spring     |
| associated with the   |     students at 24    |     2008; spring     |
| greatest increases in |     elementary        |     2009)            |
| reading               |     schools; this     |                      |
| achievement?**        |     cohort of         | -   DIBELS (spring   |
|                       |     students will be  |     2007, spring     |
|                       |     followed as first |     2008, spring     |
|                       |     and second grade  |     2009)            |
|                       |     students in years |                      |
|                       |     2 and 3 of the    | -   GORT-4 (spring   |
|                       |     study             |     2008, spring     |
|                       |                       |     2009)            |
|                       | -   Teachers in       |                      |
|                       |     Reading First and | -   TCAP scores      |
|                       |     control           |                      |
|                       |     classrooms        | -   Benchmarks       |
|                       |                       |                      |
|                       |                       | -   LOTs             |
+-----------------------+-----------------------+----------------------+
| **Are classroom       | -   Teachers in       | -   Benchmarks       |
| practices             |     Reading First and |                      |
| significantly         |     control           | -   LOTs             |
| different in Reading  |     classrooms        |                      |
| First schools         |                       | -   RFTQ and RFTQ-C  |
| compared to control   |                       |                      |
| schools?**            |                       |                      |
+-----------------------+-----------------------+----------------------+
| **Do schools with     | -   In first year of  | -   PPVT-IV (fall    |
| fully implemented     |     study, all        |     2006, spring     |
| Reading First         |     kindergarten      |     2007; spring     |
| programs have greater |     students at 24    |     2008; spring     |
| gains in reading      |     elementary        |     2009)            |
| achievement than      |     schools; this     |                      |
| those with lower      |     cohort of         | -   DIBELS (spring   |
| levels of             |     students will be  |     2007, spring     |
| implementation?**     |     followed as first |     2008, spring     |
|                       |     and second grade  |     2009)            |
|                       |     students in years |                      |
|                       |     2 and 3 of the    | -   GORT-4 (spring   |
|                       |     study             |     2008, spring     |
|                       |                       |     2009)            |
|                       | -   Teachers in       |                      |
|                       |     Reading First and | -   TCAP scores      |
|                       |     control           |                      |
|                       |     classrooms        | -   Benchmarks       |
|                       |                       |                      |
|                       |                       | -   LOTs             |
|                       |                       |                      |
|                       |                       | -   RFTQ and RFTQ-C  |
+-----------------------+-----------------------+----------------------+

**RESEARCH DESIGN AND MEASURES**

The Research Study is a three-year evaluation launched during the
2006-2007 school year. The research design employed in the study is a
multi-site replication design (Slavin & Madden, 1993). Using this
approach, each Program (Reading First)-Control school comparison is
considered a replication of the other sites involved in the study. The
design makes it possible to assess (1) the overall effects across
multiple sites on student achievement (computed as Effect Size), and (2)
the effects of particular programs replicated across sites. Finally, the
research study is longitudinal in nature, allowing comparisons to be
made over time, and providing an assessment of trends, fidelity to the
model, and the sustainability of results. The design utilizes both
quantitative and qualitative data collected from classroom observations,
teacher questionnaires, and school-developed Reading First program
implementation benchmarks. Individual student testing was conducted
during the fall of 2006 and spring of 2007 and results have been
incorporated in this report.

Site researchers were the primary data collectors in this study and were
assigned to both Reading First and control sites. The researchers were
professionals with educational backgrounds and were trained by CREP in
the fall of 2006 to conduct classroom observations and to administer
surveys to teachers. In addition to the site researchers, test
administrators were also trained by CREP to conduct individual student
assessments during the fall of 2006 and the spring of 2007. This testing
team was trained to administer both the Peabody Picture Vocabulary Test
IV (PPVT-IV) and subtests of the Dynamic Indicators of Basic Early
Literacy Skills (DIBELS).

**Sample and Participants**

The 24 schools that participated in the study consisted of 12 matched
pairs. Schools were matched on demographic and achievement data and
represent both urban and rural areas from across the state. Table 2
contains a summary of participant demographic and achievement data.

**Table 2: Reading First Research Study Participants**

<table>
<colgroup>
<col style="width: 7%" />
<col style="width: 13%" />
<col style="width: 8%" />
<col style="width: 10%" />
<col style="width: 6%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 9%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 7%" />
</colgroup>
<tbody>
<tr class="odd">
<td rowspan="2"><strong>Pair</strong></td>
<td rowspan="2"><strong>School</strong></td>
<td rowspan="2"><strong>Type</strong></td>
<td rowspan="2"><strong>District</strong></td>
<td rowspan="2"><strong>Grade</strong></td>
<td rowspan="2"><strong>Enrolled</strong></td>
<td rowspan="2"><p><strong>Econo</strong></p>
<p><strong>Disad-vantaged</strong></p></td>
<td rowspan="2"><p><strong>African</strong></p>
<p><strong>American</strong></p></td>
<td rowspan="2"><strong>White</strong></td>
<td rowspan="2"><strong>Hisp</strong></td>
<td colspan="2"><p><strong>TCAP 3<sup>rd</sup> Grade</strong></p>
<p><strong>Prof Level</strong></p></td>
</tr>
<tr class="even">
<td><strong>05-06</strong></td>
<td><strong>06-07</strong></td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 1</strong></td>
<td>Collinwood</td>
<td>Reading First</td>
<td>Wayne</td>
<td>K-4</td>
<td>361</td>
<td>70.5%</td>
<td>0.0%</td>
<td>97.8%</td>
<td>2.2%</td>
<td>84.5</td>
<td>91.4</td>
</tr>
<tr class="even">
<td>Waynesboro</td>
<td>Control</td>
<td>Wayne</td>
<td>K-4</td>
<td>415</td>
<td>70.1%</td>
<td>1.7%</td>
<td>97.6%</td>
<td>.5%</td>
<td>95.9</td>
<td>97.4</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 2</strong></td>
<td>Inskip</td>
<td>Reading First</td>
<td>Knox</td>
<td>K-5</td>
<td>399</td>
<td>90.5%</td>
<td>34.3%</td>
<td>56.1%</td>
<td>7.5%</td>
<td>76.9</td>
<td>85.0</td>
</tr>
<tr class="even">
<td>Spring Hill</td>
<td>Control</td>
<td>Knox</td>
<td>K-5</td>
<td>558</td>
<td>73.1%</td>
<td>38.0%</td>
<td>58.6%</td>
<td>2.5%</td>
<td>82.6</td>
<td>76.3</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 3</strong></td>
<td>East Lincoln</td>
<td>Reading First</td>
<td>Tullahoma</td>
<td>K-5</td>
<td>387</td>
<td>84.2%</td>
<td>10.6%</td>
<td>84.2%</td>
<td>3.1%</td>
<td>91.1</td>
<td>92.5</td>
</tr>
<tr class="even">
<td>Jack T. Farrar</td>
<td>Control</td>
<td>Tullahoma</td>
<td>K-5</td>
<td>340</td>
<td>53.1%</td>
<td>8.8%</td>
<td>88.2%</td>
<td>1.8%</td>
<td>83.6</td>
<td>94.4</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 4</strong></td>
<td>Parris South</td>
<td>Reading First</td>
<td>Hardin</td>
<td>K-5</td>
<td>475</td>
<td>72.8%</td>
<td>8.6%</td>
<td>87.2%</td>
<td>3.6%</td>
<td>92.4</td>
<td>97.8</td>
</tr>
<tr class="even">
<td>West Hardin</td>
<td>Control</td>
<td>Hardin</td>
<td>K-5</td>
<td>255</td>
<td>80.5%</td>
<td>4.3%</td>
<td>94.9%</td>
<td>0.8%</td>
<td>93.1</td>
<td>95.2</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 5</strong></td>
<td>Spring Hill</td>
<td>Reading First</td>
<td><p>Memphis</p>
<p>City</p></td>
<td>K-5</td>
<td>586</td>
<td>98.6%</td>
<td>98.3%</td>
<td>1.0%</td>
<td>0.7%</td>
<td>67.4</td>
<td>79.5</td>
</tr>
<tr class="even">
<td>Lucie E. Campbell</td>
<td>Control</td>
<td>Memphis City</td>
<td>K-6</td>
<td>756</td>
<td>97.5%</td>
<td>99.3%</td>
<td>0.7%</td>
<td>0.0</td>
<td>76.1</td>
<td>77.5</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 6</strong></td>
<td>Westhaven</td>
<td>Reading First</td>
<td>Memphis City</td>
<td>K-5</td>
<td>435</td>
<td>94.1%</td>
<td>100.0%</td>
<td>0.0</td>
<td>0.0</td>
<td>78.3</td>
<td>93.8</td>
</tr>
<tr class="even">
<td>Coleman</td>
<td>Control</td>
<td>Memphis City</td>
<td>K-5</td>
<td>619</td>
<td>91.7%</td>
<td>91.3%</td>
<td>3.6%</td>
<td>4.7%</td>
<td>75.7</td>
<td>84.3</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 7</strong></td>
<td>Westwood</td>
<td>Reading First</td>
<td>Memphis City</td>
<td>K-6</td>
<td>520</td>
<td>95.8%</td>
<td>100.0%</td>
<td>0.0</td>
<td>0.0</td>
<td>70.2</td>
<td>86.5</td>
</tr>
<tr class="even">
<td>Robert R. Church</td>
<td>Control</td>
<td>Memphis City</td>
<td>K-5</td>
<td>629</td>
<td>89.7%</td>
<td>99.7%</td>
<td>0.0</td>
<td>0.2%</td>
<td>69.8</td>
<td>83.5</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 8</strong></td>
<td>Hamilton</td>
<td>Reading First</td>
<td>Memphis City</td>
<td>K-5</td>
<td>578</td>
<td>99.2%</td>
<td>99.5%</td>
<td>0.2%</td>
<td>0.3%</td>
<td>68.8</td>
<td>81.1</td>
</tr>
<tr class="even">
<td>Fairley</td>
<td>Control</td>
<td>Memphis City</td>
<td>K-5</td>
<td>600</td>
<td>96.5%</td>
<td>100.0%</td>
<td>0.0</td>
<td>0.0</td>
<td>70.7</td>
<td>83.3</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 9</strong></td>
<td>Orchard Knob</td>
<td>Reading First</td>
<td>Hamilton</td>
<td>K-5</td>
<td>349</td>
<td>91.1%</td>
<td>99.1%</td>
<td>0.9%</td>
<td>0.0</td>
<td>60.0</td>
<td>70.2</td>
</tr>
<tr class="even">
<td>Hardy</td>
<td>Control</td>
<td>Hamilton</td>
<td>K-5</td>
<td>580</td>
<td>95.8%</td>
<td>96.6%</td>
<td>2.8%</td>
<td>0.5%</td>
<td>76.6</td>
<td>89.2</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 10</strong></td>
<td>Battle</td>
<td>Reading First</td>
<td>Hamilton</td>
<td>K-5</td>
<td>450</td>
<td>76.0%</td>
<td>76.4%</td>
<td>21.8%</td>
<td>0.7%</td>
<td>88.7</td>
<td>86.8</td>
</tr>
<tr class="even">
<td>Brown</td>
<td>Control</td>
<td>Hamilton</td>
<td>K-5</td>
<td>362</td>
<td>90.2%</td>
<td>84.8%</td>
<td>11.9%</td>
<td>2.2%</td>
<td>78.4</td>
<td>90.8</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 11</strong></td>
<td>Arlington</td>
<td>Reading First</td>
<td>Madison</td>
<td>K-4</td>
<td>483</td>
<td>91.4%</td>
<td>73.1%</td>
<td>13.7%</td>
<td>11.8%</td>
<td>82.1</td>
<td>86.5</td>
</tr>
<tr class="even">
<td>Alexander</td>
<td>Control</td>
<td>Madison</td>
<td>K-4</td>
<td>320</td>
<td>92.2%</td>
<td>90.6%</td>
<td>7.5%</td>
<td>1.9%</td>
<td>61.1</td>
<td>82.5</td>
</tr>
<tr class="odd">
<td rowspan="2"><strong>Pair 12</strong></td>
<td>North Parkway</td>
<td>Reading First</td>
<td>Madison</td>
<td>K-4</td>
<td>613</td>
<td>87.7%</td>
<td>82.9%</td>
<td>12.9%</td>
<td>2.9%</td>
<td>78.5</td>
<td>90.2</td>
</tr>
<tr class="even">
<td>Lincoln Magnet</td>
<td>Control</td>
<td>Madison</td>
<td>K-4</td>
<td>348</td>
<td>87.5%</td>
<td>81.6%</td>
<td>17.2%</td>
<td>0.9%</td>
<td>79.6</td>
<td>90.2</td>
</tr>
</tbody>
</table>

\*Source: Tennessee Department of Education:
www.k-12.state.tn.us/rptcrd06/school1asp

Kindergarten students were initially selected for participation in
individual testing. These students will be followed for the duration of
the study, thus providing a longitudinal perspective of the effects of
Reading First students as they progress through school over multiple
years. All kindergarten students from the Reading First and control
sites were invited to participate in the study and parental permission
was required.

In the 2006-2007 school year, or year one of the study, kindergarten
students were assessed twice---once in fall with the Peabody Picture
Vocabulary Test 4^th^ Edition (PPVT-IV) and again in the spring with the
PPVT-IV and subtests of DIBELS. In year two of the study, the 2007-2008
school year, first grade students from the Reading First and control
schools (prior year's kindergarten students) will be assessed in the
spring of 2008 with the PPVT-IV, DIBELS subtests, and the Gray Oral
Reading Test Fourth Edition (GORT 4). These students will be from the
original cohort of students initially tested during the 2006-2007 school
year. In year three of the study, the students (now in second grade)
will be tested with PPVT-IV and GORT 4. In years two and three of the
study, only students with previous test scores will be included in the
study. []{.mark}

**Instrumentation**

Instruments used to collect data regarding the effects of Reading First
included: (1) The Literacy Observation Tool (LOT), which was used by
independent site researchers to observe literacy instructional practices
in the classroom; (2) the teacher questionnaires developed to gather
perceptions and attitudes of teachers at RF schools and control schools;
and (3) the implementation benchmarking tool, used to assess progress
towards goals---used for Reading First schools only. In addition, during
the first year of the study, two instruments were used to collect
quantitative data on student achievement. These
individually-administered reading tests included the Peabody Picture
Vocabulary Test (PPVT-IV) and the Dynamic Indicators of Basic Early
Literacy Skills (DIBELS). Table 3 summarizes the data collection.

**Table 3: Data Collection Summary**

+-----------+-----------------+--------+-----------------------------+
| **Type of | **Instruments** | **Time | **Description**             |
| Measure** |                 | line** |                             |
+-----------+-----------------+--------+-----------------------------+
| Classroom | > LOT           | Fall   | -   60 received (Reading    |
|           |                 | 2006 - |     First, treatment)       |
| Obs       |                 | Spring |                             |
| ervations |                 | 2007   | -   59 received (control)   |
|           |                 |        |                             |
|           |                 |        | -   Conducted by site       |
|           |                 |        |     researchers             |
+-----------+-----------------+--------+-----------------------------+
| Surveys   | > RFTQ          | Spring | -   262 received            |
|           | >               | 2006   |     (treatment).            |
|           | > RFTQ (C)      |        |     Administered by         |
|           |                 |        |     Literacy Leaders or     |
|           |                 |        |     taken online by         |
|           |                 |        |     teachers                |
|           |                 |        |                             |
|           |                 |        | -   189 received (control). |
|           |                 |        |     Administered by site    |
|           |                 |        |     researchers             |
+-----------+-----------------+--------+-----------------------------+
| Imple     | > Benchmarking  | During | -   Completed by 12 Reading |
| mentation | > Tool          | the    |     First schools to        |
|           |                 | 200    |     indicate progress       |
| Ben       |                 | 6-2007 |     towards Reading First   |
| chmarking |                 | school |     goals.                  |
|           |                 | year   |                             |
+-----------+-----------------+--------+-----------------------------+
| Ind       | > PPVT-IV --RF  | Fall   | -   861 used in statistical |
| ividually | >               | 2006 & |     analyses (Reading       |
|           | > PPVT-IV       | Spring |     First, treatment).      |
| Adm       | > --Control     | 2007   |                             |
| inistered | >               |        | -   855 used in statistical |
| Tests     | > DIBELS        |        |     analyses (control)      |
|           | > subtests-RF   |        |                             |
|           | >               |        | -   861 used in statistical |
|           | > DIBELS        |        |     analyses (Reading       |
|           | > subtests      |        |     First, treatment).      |
|           | > --Control     |        |                             |
|           |                 |        | -   855 used in statistical |
|           |                 |        |     analyses (control)      |
+-----------+-----------------+--------+-----------------------------+

**Literacy Observation Tool (LOT^©^)**

The LOT was developed by researchers at the Center for Research in
Educational Policy, College of Education, The University of Memphis
(Smith, Ross, & Grehan, 2002). It is an instrument for observing
teachers' instructional practices in K-3 classrooms where teachers are
engaged in teaching reading and using other reading materials (e.g.,
reading centers, manipulatives, etc.). The LOT was designed to assist
schools in evaluating the effectiveness of teacher implementation of
research-based reading strategies and practices.

The standard LOT procedure involves a trained observer visiting 7-9
randomly selected classrooms, for 10 minutes each, during the typical 1½
to 2 hour reading/ literacy block. The LOT was piloted in the Memphis
City Schools and is used in multiple schools across the United States,
including 75 Reading First schools in the state of Tennessee. In this
research study, site researchers conducted 5 LOTs at each treatment and
each control school during the 2006-2007 school year.

**Teacher Surveys**

**Reading First Teacher Questionnaire (RFTQ)**

All K-3 teachers and special education teachers who taught reading were
asked to complete the RFTQ, which contains 27 items to which teachers
responded using a five-point Likert-type scale that ranges from (1)
Strongly Disagree to (5) Strongly Agree. Items assessed included the
specific program elements of Reading First such as: general impressions
of professional development, support provided by the Literacy Leader and
the Reading Cadre, teacher support for the program, impacts on student
achievement, changes in teaching and assessment, and understanding of
the Reading First program. In addition, one open-ended question asked
teachers to respond to the statement: Please provide any additional
comments you may have pertaining to your school's Reading First program.
A final section of the RFTQ contained demographic questions regarding
years of teaching experience, age, gender, race, and education. Teachers
in Reading First schools took the RFTQ on-line, and there were 262
respondents.

**Reading First Teacher Questionnaire-Control (RFTQ-C)**

A second questionnaire (RFTQ-C) was developed for use in the comparison
schools. This instrument was patterned after the RFTQ. Because
comparison schools did not receive Reading First funds, all references
in the instrument to Reading First were replaced with phrases such as
"our school's literacy program" in the RFTQ-C. As with the RFTQ, all K-3
teachers and special education teachers who taught reading were asked to
complete the RFTQ-C, which contained 27 items to which teachers
responded using a five-point Likert-type scale that ranged from (1)
Strongly Disagree to (5) Strongly Agree. In addition, one open-ended
question asked teachers to respond to the statement: Please provide any
additional comments you may have pertaining to your school's reading
program. A final section of the RFTQ-C contained demographic questions
regarding years of teaching experience, age, gender, race, and
education. Site researchers administered the questionnaire in the spring
to the control teachers and there were 189 teachers who responded.

**Implementation Benchmarking Tool**

The Implementation Benchmarking Tool was developed by CREP. Generic RF
benchmarks were developed by CREP to assist schools in refining and
further developing each school's individual benchmarks directly geared
to their school's implementation of the RF grant. The five components of
effective reading instruction and RF assessments were included in the
generic model.

As part of the benchmark development process, the Literacy Leader and a
designated school benchmark team developed statements or goals for each
major program component. Each statement was then elaborated by specific
*indicator* and *evidence* for implementation phases (Phase I =
Beginning; Phase II = Intermediate, and Phase III = Full
Implementation). Typical benchmarking timelines are to complete the
implementation benchmarks by mid-October, refine them if needed during
the year, and then, by early May, engage the benchmark team in
evaluating progress and specifying program goals for the following year.
Based on those goals, the benchmarks are continually revised and
refined. Consequently, participating schools are continually aware of
all program components, implementation progress, directions for school
improvement directed by data, and shared faculty-administrator
decision-making.

For the purposes of this study, only treatment schools used the program
benchmarks and differences between the 12 treatment schools were
examined.

**Literacy Tests**

*Peabody Picture Vocabulary Test- 4^th^ edition*. The Peabody Picture
Vocabulary Test (PPVT-IV) (Dunn & Dunn, 2006) is the most recent version
of this popular assessment of receptive vocabulary. Like the previous
editions, the PPVT-IV is administered to individual students and is a
norm-referenced assessment test of receptive vocabulary that measures
listening comprehension of spoken words for children. The PPVT-IV is
often used because of the importance of vocabulary as a measure of child
development and because the test is easy to administer to young
children. The PPVT-IV has several noteworthy improvements: it has been
colorized to make it more appealing to children; and it contains an
increase in the number of items to improve the accuracy of the test. The
age-norm and grade-norm samples were designed to resemble the
English-proficient population from ages 2.6 years to 90+ years and to
closely match the 2004 census data in demographics. The test was normed
in 2005 and 2006 and included 450 examiners from 320 test sites. The age
norm included 3,540 cases and the grade subsample was 2003 cases and
included grades kindergarten through 12. Internal consistency was .94
and .93 for test-retest.

*Dynamic Indicators of Basic Early Literacy Skills*. The Dynamic
Indicators of Basic Early Literacy Skills (DIBELS) is a tool for early
identification of children with potential literacy problems and an
assessment of response to instruction. The DIBELS assessment is designed
to enable educators to modify their approach if a student is not on
course to achieve reading goals. The Early Childhood Research Institute
on Measuring Growth and Development at the University of Oregon
constructed DIBELS. The measures were developed based upon the essential
early literacy domains discussed in both the National Reading Panel
(2000) and National Research Council (1998) reports to assess student
development of phonological awareness, alphabetic understanding, and
automaticity and fluency with the code. Each measure has been thoroughly
researched and demonstrated to be reliable and valid indicators of early
literacy development and predictive of later reading proficiency to aid
in the early identification of students who are not progressing as
expected. When used as recommended, the results can be used to evaluate
individual student development as well as provide grade-level feedback
toward validated instructional objectives**.** The Institute has
validated the instrument's ability to predict outcomes and has tested
its reliability with young children across the country.

For the kindergarten students in this study, subtests of DIBELS that
were given included the following:

-   **Letter Naming Fluency (LNF):** intended for most children from
    fall of kindergarten through fall of first grade. Students are
    presented with a page of uppercase and lowercase letters arranged in
    a random order and are asked to name as many letters as they can.
    The student is allowed one minute to produce as many letter names as
    he/she can, and the score is the number of letters named correctly
    in one minute.

-   **Phoneme Segmentation Fluency (PSF**): intended for most children
    from winter of kindergarten through spring of first grade. This
    subtest is designed to assess a student's ability to fluently
    segment three and four-phoneme words into their individual phonemes.
    The tester presents the word to the student, who verbally responds
    with the correct phoneme breakdown of the word.

```{=html}
<!-- -->
```
-   **Nonsense Word Fluency (NWF):** intended for most children from mid
    to end of kindergarten through the beginning of second grade. The
    subtest is designed to test student's knowledge of alphabetic
    principle and letter-sound correspondence; as well as the ability to
    blend letters into words in which letters represent their most
    common sounds. The student is given a sheet of paper with randomly
    ordered vowel-consonant or consonant-vowel-consonant nonsense words
    and asked to verbalize the individual letter sound of each word or
    "read" the nonsense word. Students who "read" the nonsense word
    rather than simply provide the letter sound in isolation receive a
    higher score.

**PROCEDURE**

During the summer of 2006, the State of Tennessee Department of
Education provided CREP with a list of all schools that were recipients
of Reading First funds. A total of twelve schools were selected to
participate in the research study. A stratified random sampling
technique based on urban and rural locations across the state was used
to provide geographic representativeness. Reading First schools were
then matched to comparison schools, which were selected on the basis of
similar school-level demographic, academic, and geographic background
variables. (These comparisons were presented earlier in Table 2).
Feedback from the state Department of Education and individual school
districts was provided to further refine the selection process.

In 2006, 11 site observers were selected to conduct observations for the
research study (both Reading First and control schools). During the
early fall, these site researchers participated in LOT training, which
included practice observations in actual classrooms, as well as peer
consensus ratings for the purpose of interrater reliability. Site
researchers conducted a total of five observations at each school
throughout the fall and spring semesters. Generally, observations were
scheduled a month apart and distributed throughout the days of the week,
for a complete representation of K-3 reading instruction. Because the
reading block and literacy instruction typically occurred during the
morning, site researchers observed classrooms at that time of day. LOT
observations were mailed directly to CREP where quality assurance
procedures were conducted. For control schools, site researchers were
also responsible for administering the RFTQ-C survey to teachers at the
control sites; Reading First teachers were able to complete their
surveys on-line.

Test administrators were selected and trained to conduct individual
student testing in the fall of 2006 and the spring of 2007. Training
sessions incorporated the viewing of videotapes of test administration,
modeling by the instructor, and practice administration with students.
In the fall, test administrators visited each school to test individual
kindergarten students who had parental permission to participate in the
study. In the spring, test administrators again visited the schools and
tested those children that had been previously tested in the fall.
Testing typically lasted one full day at each school with testers
administering the tests in a quiet location in the school building. A
supervisor was present to provide quality control and a staff member
inspected every test shortly after each administration. Testing at both
treatment and control schools was completed during a three week period.
Tests were then scored, entered into the database, and subjected to a
final round of quality assurance.

**Results**

**Classroom Observation Measure Results**

**LOT**

***Descriptive Results***

**Instructional Orientation**

In both the Reading First classrooms and the control classrooms, the
most frequently observed form of instructional orientation was whole
class instruction. This was observed either frequently or extensively in
63.4% of the Reading First observations and in 88.1% of the control
classrooms. However, Reading First sites were observed utilizing more
variety in their instructional approach than their control counterparts,
with learning centers observed frequently or extensively 20% of the time
(3.4% for control sites); and small group instruction observed
frequently or extensively in 16.7% of the observations (compared with
3.4% for the control classrooms. Cooperative/ collaborative learning was
not frequently or extensively observed in control classrooms; Reading
First sites were observed utilizing this orientation in 6.7% of the
observations. (See Table 4 for complete data summary from the LOT.)

**Table 4: Literacy Observation Tool (LOT^©^): Year 1 (2006-2007)**

**Reading First, Treatment Schools (T) N= 60**

**Control Schools (C) N= 59**

<table style="width:100%;">
<colgroup>
<col style="width: 33%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
</colgroup>
<tbody>
<tr class="odd">
<td></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Not Observed</strong></p></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Rarely</strong></p>
<p><strong>Observed</strong></p></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Occasionally</strong></p>
<p><strong>Observed</strong></p></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Frequently</strong></p>
<p><strong>Observed</strong></p></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Extensively</strong></p>
<p><strong>Observed</strong></p></td>
</tr>
<tr class="even">
<td></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
</tr>
<tr class="odd">
<td colspan="3"><strong>INSTRUCTIONAL ORIENTATION</strong></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td>Small group</td>
<td>10.0</td>
<td>28.8</td>
<td>38.3</td>
<td>42.4</td>
<td>35.0</td>
<td>25.4</td>
<td>10.0</td>
<td>3.4</td>
<td>6.7</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td>Whole class</td>
<td>1.7</td>
<td>0.0</td>
<td>11.7</td>
<td>1.7</td>
<td>21.7</td>
<td>10.2</td>
<td>41.7</td>
<td>35.6</td>
<td>21.7</td>
<td>52.5</td>
</tr>
<tr class="even">
<td>Learning centers</td>
<td>21.7</td>
<td>45.8</td>
<td>31.7</td>
<td>40.7</td>
<td>26.7</td>
<td>10.2</td>
<td>16.7</td>
<td>3.4</td>
<td>3.3</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td>Cooperative/collaborative Learning</td>
<td>35.0</td>
<td>55.9</td>
<td>36.7</td>
<td>40.7</td>
<td>18.3</td>
<td>3.4</td>
<td>5.0</td>
<td>0.0</td>
<td>1.7</td>
<td>0.0</td>
</tr>
<tr class="even">
<td colspan="3"><strong>INSTRUCTIONAL COMPONENTS</strong></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td><em><strong><u>Reading - The Teacher:</u></strong></em></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><em><strong>Concepts of Print</strong></em></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Book/print conventions</td>
<td>30.0</td>
<td>37.3</td>
<td>50.0</td>
<td>37.3</td>
<td>16.7</td>
<td>15.3</td>
<td>1.7</td>
<td>6.8</td>
<td>1.7</td>
<td>3.4</td>
</tr>
<tr class="even">
<td><em><strong>Alphabetics</strong></em></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Letter naming/knowledge</td>
<td>15.0</td>
<td>18.6</td>
<td>61.7</td>
<td>54.2</td>
<td>13.3</td>
<td>20.3</td>
<td>5.0</td>
<td>6.8</td>
<td>3.3</td>
<td>0.0</td>
</tr>
<tr class="even">
<td>Explicit phonemic awareness instruction</td>
<td>21.7</td>
<td>39.0</td>
<td>35.0</td>
<td>44.1</td>
<td>31.7</td>
<td>15.3</td>
<td>11.7</td>
<td>1.7</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td>Explicit phonics instruction</td>
<td>13.3</td>
<td>32.2</td>
<td>53.3</td>
<td>49.2</td>
<td>16.7</td>
<td>15.3</td>
<td>15.0</td>
<td>3.4</td>
<td>1.7</td>
<td>0.0</td>
</tr>
<tr class="even">
<td><em><strong>Fluency</strong></em></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Models fluent oral reading</td>
<td>6.7</td>
<td>10.2</td>
<td>21.7</td>
<td>30.5</td>
<td>31.7</td>
<td>25.4</td>
<td>33.3</td>
<td>25.4</td>
<td>5.0</td>
<td>8.5</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Has students read/reread orally together</p>
</blockquote></td>
<td>6.7</td>
<td>13.6</td>
<td>21.7</td>
<td>35.6</td>
<td>26.7</td>
<td>25.4</td>
<td>35.0</td>
<td>16.9</td>
<td>8.3</td>
<td>8.5</td>
</tr>
<tr class="odd">
<td><em><strong>Vocabulary</strong></em></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p>Introduces/reviews key vocabulary</p>
</blockquote></td>
<td>3.3</td>
<td>5.1</td>
<td>30.0</td>
<td>32.2</td>
<td>25.0</td>
<td>27.1</td>
<td>23.3</td>
<td>25.4</td>
<td>18.3</td>
<td>10.2</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Explicit vocabulary instruction</p>
</blockquote></td>
<td>30.0</td>
<td>52.5</td>
<td>36.7</td>
<td>35.6</td>
<td>11.7</td>
<td>11.9</td>
<td>16.7</td>
<td>0.0</td>
<td>3.3</td>
<td>0.0</td>
</tr>
<tr class="even">
<td><em><strong>Text Comprehension</strong></em></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Explicit comprehension strategy instruction</p>
</blockquote></td>
<td>18.3</td>
<td>18.6</td>
<td>31.7</td>
<td>47.5</td>
<td>16.7</td>
<td>25.4</td>
<td>18.3</td>
<td>6.8</td>
<td>15.0</td>
<td>1.7</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Makes connection to prior knowledge</p>
</blockquote></td>
<td>16.7</td>
<td>16.9</td>
<td>33.3</td>
<td>45.8</td>
<td>23.3</td>
<td>15.3</td>
<td>11.7</td>
<td>18.6</td>
<td>15.0</td>
<td>3.4</td>
</tr>
<tr class="odd">
<td>Asks students for predictions</td>
<td>25.0</td>
<td>30.5</td>
<td>38.3</td>
<td>33.9</td>
<td>15.0</td>
<td>18.6</td>
<td>13.3</td>
<td>16.9</td>
<td>8.3</td>
<td>0.0</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Uses higher level questioning</p>
</blockquote></td>
<td>25.0</td>
<td>39.0</td>
<td>40.0</td>
<td>28.8</td>
<td>11.7</td>
<td>15.3</td>
<td>8.3</td>
<td>13.6</td>
<td>15.0</td>
<td>3.4</td>
</tr>
<tr class="odd">
<td>Guides visual imaging</td>
<td>43.3</td>
<td>49.2</td>
<td>38.3</td>
<td>40.7</td>
<td>5.0</td>
<td>8.5</td>
<td>8.3</td>
<td>1.7</td>
<td>3.3</td>
<td>0.0</td>
</tr>
<tr class="even">
<td>Guides interactive discussion</td>
<td>36.7</td>
<td>44.1</td>
<td>23.3</td>
<td>25.4</td>
<td>21.7</td>
<td>20.3</td>
<td>10.0</td>
<td>5.1</td>
<td>8.3</td>
<td>5.1</td>
</tr>
</tbody>
</table>

**Table 4: Continued**

<table style="width:100%;">
<colgroup>
<col style="width: 33%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 5%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
<col style="width: 6%" />
</colgroup>
<tbody>
<tr class="odd">
<td></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Not Observed</strong></p></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Rarely</strong></p>
<p><strong>Observed</strong></p></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Occasionally</strong></p>
<p><strong>Observed</strong></p></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Frequently</strong></p>
<p><strong>Observed</strong></p></td>
<td colspan="2"><p><strong>Percent</strong></p>
<p><strong>Extensively</strong></p>
<p><strong>Observed</strong></p></td>
</tr>
<tr class="even">
<td></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
<td><strong>T</strong></td>
<td><strong>C</strong></td>
</tr>
<tr class="odd">
<td><strong>STUDENT ACTIVITIES</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Reads self-selected materials</td>
<td>56.7</td>
<td>64.4</td>
<td>35.0</td>
<td>33.9</td>
<td>5.0</td>
<td>1.7</td>
<td>1.7</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Writes independently in response to reading</p>
</blockquote></td>
<td>53.3</td>
<td>69.5</td>
<td>36.7</td>
<td>28.8</td>
<td>8.3</td>
<td>1.7</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="even">
<td>Formal testing/assessment</td>
<td>80.0</td>
<td>81.4</td>
<td>16.7</td>
<td>18.6</td>
<td>3.3</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td><strong>LEARNING ENVIRONMENT</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p>Conducive to cooperative interactions</p>
</blockquote></td>
<td>0.0</td>
<td>0.0</td>
<td>1.7</td>
<td>5.1</td>
<td>6.7</td>
<td>6.8</td>
<td>5.0</td>
<td>8.5</td>
<td>85.0</td>
<td>79.7</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Students actively engaged</p>
</blockquote></td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1.7</td>
<td>10.2</td>
<td>98.3</td>
<td>89.8</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Effective classroom management</p>
</blockquote></td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1.7</td>
<td>3.4</td>
<td>3.3</td>
<td>11.9</td>
<td>95.0</td>
<td>84.7</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Teacher actively monitors</p>
</blockquote></td>
<td>0.0</td>
<td>1.7</td>
<td>3.3</td>
<td>0.0</td>
<td>0.0</td>
<td>3.4</td>
<td>5.0</td>
<td>8.5</td>
<td>91.7</td>
<td>86.4</td>
</tr>
<tr class="even">
<td><strong>VISIBLE PRINT ENVIRONMENT</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Alphabet</p>
</blockquote></td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1.7</td>
<td>0.0</td>
<td>0.0</td>
<td>3.3</td>
<td>5.1</td>
<td>96.7</td>
<td>93.2</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Word wall</p>
</blockquote></td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1.7</td>
<td>0.0</td>
<td>1.7</td>
<td>5.0</td>
<td>8.5</td>
<td>95.0</td>
<td>88.1</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Classroom library</p>
</blockquote></td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1.7</td>
<td>0.0</td>
<td>0.0</td>
<td>3.3</td>
<td>1.7</td>
<td>96.7</td>
<td>96.6</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Evidence of student writing/work products</p>
</blockquote></td>
<td>0.0</td>
<td>5.1</td>
<td>10.0</td>
<td>13.6</td>
<td>18.3</td>
<td>20.3</td>
<td>13.3</td>
<td>22.0</td>
<td>58.3</td>
<td>39.0</td>
</tr>
<tr class="odd">
<td><strong>MATERIALS USED</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p>Basal texts</p>
</blockquote></td>
<td>1.7</td>
<td>16.9</td>
<td>23.3</td>
<td>32.2</td>
<td>43.3</td>
<td>28.8</td>
<td>28.3</td>
<td>22.0</td>
<td>3.3</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Big books</p>
</blockquote></td>
<td>48.3</td>
<td>59.3</td>
<td>43.3</td>
<td>40.7</td>
<td>6.7</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Books on tape</p>
</blockquote></td>
<td>55.0</td>
<td>67.8</td>
<td>35.0</td>
<td>30.5</td>
<td>10.0</td>
<td>1.7</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Computers</p>
</blockquote></td>
<td>33.3</td>
<td>39.0</td>
<td>25.0</td>
<td>42.4</td>
<td>30.0</td>
<td>18.6</td>
<td>11.7</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Fiction books</p>
</blockquote></td>
<td>35.0</td>
<td>33.9</td>
<td>35.0</td>
<td>44.1</td>
<td>16.7</td>
<td>6.8</td>
<td>10.0</td>
<td>15.3</td>
<td>3.3</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Non-fiction books</p>
</blockquote></td>
<td>65.0</td>
<td>81.4</td>
<td>23.3</td>
<td>18.6</td>
<td>11.7</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Poetry</p>
</blockquote></td>
<td>73.3</td>
<td>79.7</td>
<td>25.0</td>
<td>18.6</td>
<td>0.0</td>
<td>1.7</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Newspaper/magazines</p>
</blockquote></td>
<td>93.3</td>
<td>89.8</td>
<td>6.7</td>
<td>10.2</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Word/vocabulary materials</p>
</blockquote></td>
<td>5.0</td>
<td>11.9</td>
<td>23.3</td>
<td>45.8</td>
<td>38.3</td>
<td>27.1</td>
<td>26.7</td>
<td>10.2</td>
<td>6.7</td>
<td>5.1</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Worksheets/workbooks</p>
</blockquote></td>
<td>8.3</td>
<td>5.1</td>
<td>40.0</td>
<td>50.8</td>
<td>30.0</td>
<td>37.3</td>
<td>20.0</td>
<td>6.8</td>
<td>1.7</td>
<td>0.0</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Student notebooks/journals</p>
</blockquote></td>
<td>55.0</td>
<td>57.6</td>
<td>35.0</td>
<td>32.2</td>
<td>6.7</td>
<td>6.8</td>
<td>3.3</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Chart tablets/whiteboards</p>
</blockquote></td>
<td>11.7</td>
<td>10.2</td>
<td>25.0</td>
<td>42.4</td>
<td>26.7</td>
<td>25.4</td>
<td>33.3</td>
<td>18.6</td>
<td>3.3</td>
<td>3.4</td>
</tr>
</tbody>
</table>

### Instructional Components

LOT observations indicated a greater variety of instructional components
addressed and a wider array of instructional techniques in Reading First
classrooms than in the control classrooms. While both fluency and
vocabulary components were notably present in both Reading First and
control sites, observers noted Reading First teachers frequently or
extensively instructed students in oral reading in 43.3% of the
observations, while for control sites this occurred during 24.5% of the
observations. Also, Reading First teachers modeled fluent oral reading
either frequently or extensively during 38.3% of the visits; for control
sites this was noted during 33.9% of the visits. Data suggested Reading
First teachers used text comprehension components to a greater extent
than control teachers. Most noticeably, explicit comprehension
instruction was observed frequently or extensively in 33.3% of the
Reading First classrooms, while control teachers were frequently or
extensively observed using this strategy in 8.5% of the observations. In
all categories of text comprehension, Reading First teachers were
observed frequently or extensively utilizing the components to a greater
degree than the control teachers.

Alphabetic instructional techniques in general were not observed as
frequently as other techniques in either the Reading First sites or the
control sites. Of the alphabetics techniques employed, explicit phonics
instruction was frequently or extensively observed in 16.7% of the
treatment classrooms; for control classrooms, this was noted in 3.4% of
the observations. Explicit phonemic awareness instruction was frequently
observed in 11.7 % of the Reading First classes; for control classrooms
this was frequently observed 1.7% of the time.

Concepts of print was the only component which was more frequently or
extensively observed in control classrooms. Book/print conventions were
frequently or extensively observed in 10.2% of the control classrooms;
for Reading First classrooms, only 3.4% of the observers noted this.

### Student Activities

The majority of observations reflected students in both Reading First
and control schools were rarely or never engaged in the following:
reading self-selected materials; writing independently in response to
reading; or engaging in formal testing. Treatment students were
frequently or occasionally observed reading self-selected materials
during 6.7% of the visits and occasionally writing independently in
response to reading in 8.3% of the visits.

### Learning Environment

The observation data suggested positive learning environments in both
treatment and control classrooms. In over 85% of the observations,
observers noted frequent or extensive: conduciveness to cooperative
interactions, students actively engaged, effective classroom management,
and teachers actively monitoring.

### Visible Print Environment

A print rich environment was observed in both treatment and control
classrooms. Alphabet displays, visible world walls, and accessible
classroom libraries were found either extensively or frequently in over
95% of all classrooms. Evidence of student writing and work products
were less frequently observed in Reading First and control classrooms.
Over seventy percent of the Reading First classes (71.6%) demonstrated
evidence of this; while for control sites, 61% of the observations
reflected frequent or extensive display of student writing.

### Materials Used

LOT observations indicated that Reading First teachers used a wider
variety of materials in their classrooms than the control teachers did.
In treatment classrooms, researchers noted frequent of extensive use of:
chart tablets/white boards (36.6%), word/vocabulary materials (33.4%),
basal texts (31.6%), worksheets/workbooks (21.7%), fiction books
(13.3%), computers (11.7%), and student notebooks (3.3%). In the control
classrooms, teachers were frequently or extensively observed utilizing:
basal texts (22%), chart tables/ whiteboards (22%), fiction books
(15.3%), word/vocabulary materials (15.3%) and worksheets (6.8%).

***Inferential Results***

The 41 items on the LOT are all measured on the same 5-point
Likert-scale: 0=Not observed, 1=Rarely, 2=Occasionally, 3=Frequently and
4=Extensively. As a result, the alpha level was set to a Bonferroni
adjusted .0012 (.05 divided by 41 questions). The number of returned LOT
observations was almost identical for the treatment group (N=60) and the
control group (N=59). The Wilcoxon Mann Whitney was conducted for the
LOT.

Only three out of 41 items were found to be statistically significant
between the two groups. As seen in Table 5, "Small group" and "Learning
centers" in the Instructional Orientation section had a higher mean and
mean rank score for treatment schools and were observed significantly
more frequently, while "Whole class" (also in the Instructional
Orientation section) had a higher mean and mean rank score and was
observed significantly more frequently for the control group. For the
remaining 38 LOT items, while not statistically significant, the
majority exhibited a higher mean and mean rank score in the treatment
schools relative to the control schools, revealing a higher frequency in
the treatment schools.

**Table 5: LOT Items with Significant Differences Between Treatment and
Control Groups**

+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
|                       | *   |     |     | **  |     |     | *  | *  |
|                       | *Tr |     |     | Con |     |     | ** | ** |
|                       | eat |     |     | tro |     |     | z* | p* |
|                       | men |     |     | l** |     |     | ** | ** |
|                       | t** |     |     |     |     |     |    |    |
+=======================+=====+=====+=====+=====+=====+=====+====+====+
|                       | *   | **  | *   | *   | **  | *   |    |    |
|                       | **M | *SD | **M | **M | *SD | **M |    |    |
|                       | *** | *** | ean | *** | *** | ean |    |    |
|                       |     |     | R   |     |     | R   |    |    |
|                       |     |     | ank |     |     | ank |    |    |
|                       |     |     | Sc  |     |     | Sc  |    |    |
|                       |     |     | ore |     |     | ore |    |    |
|                       |     |     | *** |     |     | *** |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > **INSTRUCTIONAL     |     |     |     |     |     |     |    |    |
| > ORIENTATION**       |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Small group           | 1   | 1   | 69  | 1   | 0   | 50  | -  | .0 |
|                       | .65 | .02 | .72 | .03 | .83 | .12 | 3. | 01 |
|                       |     |     |     |     |     |     | 26 |    |
|                       |     |     |     |     |     |     | \* |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Whole class           | 2   | 1   | 47  | 3   | 0   | 71  | 3. | \< |
|                       | .71 | .00 | .94 | .39 | .74 | .06 | 90 | .0 |
|                       |     |     |     |     |     |     | \* | 01 |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Learning centers      | 1   | 1   | 71  | 0   | 0   | 47  | -  | \< |
|                       | .48 | .11 | .81 | .71 | .79 | .99 | 3. | .0 |
|                       |     |     |     |     |     |     | 95 | 01 |
|                       |     |     |     |     |     |     | \* |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Coop                | 0   | 0   | 67  | 0   | 0   | 50  | 2. | .0 |
| erative/collaborative | .98 | .96 | .61 | .47 | .57 | .53 | 97 | 03 |
| > learning            |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > **INSTRUCTIONAL     |     |     |     |     |     |     |    |    |
| > COMPONENTS**        |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***[Reading - The   |     |     |     |     |     |     |    |    |
| > Tea                 |     |     |     |     |     |     |    |    |
| cher:]{.underline}*** |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***Concepts of      |     |     |     |     |     |     |    |    |
| > Print***            |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Book/print          | 0   | 0   | 60  | 1   | 1   | 59  | -  | .9 |
| > Conventions         | .95 | .83 | .27 | .02 | .06 | .73 | 0. | 30 |
|                       |     |     |     |     |     |     | 09 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***Alphabetic***    |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Letter              | 1   | 0   | 59  | 1   | 0   | 59  | 0. | .9 |
| > naming/knowledge    | .19 | .88 | .31 | .15 | .81 | .69 | 06 | 49 |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Explicit phonemic   | 1   | 0   | 69  | 0   | 0   | 50  | -  | .0 |
| > awareness           | .33 | .95 | .36 | .80 | .76 | .48 | 3. | 02 |
| > Instruction         |     |     |     |     |     |     | 15 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Explicit phonics      | 1   | 0   | 68  | 0   | 0   | 51  | -  | .0 |
| instruction           | .38 | .96 | .07 | .90 | .78 | .80 | 2. | 05 |
|                       |     |     |     |     |     |     | 79 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***Fluency***       |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Models fluent oral    | 2   | 1   | 62  | 1   | 1   | 56  | -  | .3 |
| reading               | .08 | .02 | .29 | .92 | .15 | .71 | 0. | 60 |
|                       |     |     |     |     |     |     | 91 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Has students        | 2   | 1   | 66  | 1   | 1   | 52  | -  | .0 |
| > read/reread orally  | .17 | .09 | .47 | .71 | .16 | .53 | 2. | 22 |
| > together            |     |     |     |     |     |     | 28 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+

**Table 5: Continued**

+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
|                       | *   |     |     | **  |     |     | *  | *  |
|                       | *Tr |     |     | Con |     |     | ** | ** |
|                       | eat |     |     | tro |     |     | z* | p* |
|                       | men |     |     | l** |     |     | ** | ** |
|                       | t** |     |     |     |     |     |    |    |
+=======================+=====+=====+=====+=====+=====+=====+====+====+
|                       | *   | **  | *   | *   | **  | *   |    |    |
|                       | **M | *SD | **M | **M | *SD | **M |    |    |
|                       | *** | *** | ean | *** | *** | ean |    |    |
|                       |     |     | R   |     |     | R   |    |    |
|                       |     |     | ank |     |     | ank |    |    |
|                       |     |     | Sc  |     |     | Sc  |    |    |
|                       |     |     | ore |     |     | ore |    |    |
|                       |     |     | *** |     |     | *** |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***Vocabulary***    |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Introduces/reviews  | 2   | 1   | 62  | 2   | 1   | 57  | -  | .3 |
| > key vocabulary      | .23 | .17 | .68 | .03 | .10 | .27 | 0. | 78 |
|                       |     |     |     |     |     |     | 88 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Explicit vocabulary | 1   | 1   | 68  | 0   | 0   | 50  | -  | .0 |
| > instruction         | .25 | .17 | .79 | .59 | .70 | .21 | 3. | 02 |
|                       |     |     |     |     |     |     | 15 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***Text             |     |     |     |     |     |     |    |    |
| > Comprehension***    |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Explicit            | 1   | 1   | 66  | 1   | 0   | 53  | -  | .0 |
| > comprehension       | .80 | .35 | .22 | .25 | .90 | .68 | 2. | 39 |
| > strategy            |     |     |     |     |     |     | 06 |    |
| > instruction         |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Makes connection to | 1   | 1   | 63  | 1   | 1   | 56  | -  | .2 |
| > prior knowledge     | .75 | .30 | .56 | .46 | .09 | .38 | 1. | 39 |
|                       |     |     |     |     |     |     | 18 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Asks students for     | 1   | 1   | 62  | 1   | 1   | 57  | -  | .4 |
| predictions           | .42 | .24 | .13 | .22 | .07 | .84 | 0. | 82 |
|                       |     |     |     |     |     |     | 70 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Uses higher level   | 1   | 1   | 64  | 1   | 1   | 55  | -  | .1 |
| > questioning         | .48 | .36 | .28 | .14 | .18 | .64 | 1. | 56 |
|                       |     |     |     |     |     |     | 42 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Guides visual imaging | 0   | 1   | 62  | 0   | 0   | 56  | -0 | .3 |
|                       | .88 | .07 | .15 | .63 | .72 | .85 | .9 | 58 |
|                       |     |     |     |     |     |     | 19 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Guides interactive    | 1   | 1   | 63  | 1   | 1   | 56  | -  | .2 |
| discussion            | .30 | .29 | .51 | .02 | .15 | .43 | 1. | 42 |
|                       |     |     |     |     |     |     | 17 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***STUDENT          |     |     |     |     |     |     |    |    |
| > ACTIVITIES***       |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Reads self-selected   | 0   | 0   | 62  | 0   | 0   | 56  | -  | .3 |
| materials             | .51 | .68 | .01 | .37 | .52 | .99 | .9 | 53 |
|                       |     |     |     |     |     |     | 29 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Writes              | 0   | 0   | 64  | 0   | 0   | 54  | -  | .0 |
| > independently in    | .54 | .65 | .53 | .32 | .51 | .47 | 1. | 61 |
| > response to reading |     |     |     |     |     |     | 87 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| Formal                | 0   | 0   | 60  | 0   | 0   | 59  | -  | .7 |
| testing/assessment    | .23 | .50 | .58 | .19 | .39 | .41 | 0. | 89 |
|                       |     |     |     |     |     |     | 27 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***LEARNING         |     |     |     |     |     |     |    |    |
| > ENVIRONMENT***      |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Conducive to        | 3   | 0   | 61  | 3   | 0   | 57  | -  | .3 |
| > cooperative         | .76 | .65 | .53 | .63 | .83 | .47 | 0. | 25 |
| > interactions        |     |     |     |     |     |     | 98 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Students actively   | 3   | 0   | 62  | 3   | 0   | 57  | -  | .0 |
| > engaged             | .98 | .13 | .51 | .90 | .30 | .45 | 1. | 50 |
|                       |     |     |     |     |     |     | 96 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Effective classroom | 3   | 0   | 63  | 3   | 0   | 56  | -  | .0 |
| > management          | .93 | .31 | .00 | .81 | .47 | .95 | 1. | 68 |
|                       |     |     |     |     |     |     | 83 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Teacher actively    | 3   | 0   | 61  | 3   | 0   | 58  | -  | .3 |
| > monitors            | .85 | .58 | .52 | .78 | .67 | .46 | 0. | 74 |
|                       |     |     |     |     |     |     | 89 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***VISIBLE PRINT    |     |     |     |     |     |     |    |    |
| > ENVIRONMENT***      |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Alphabet            | 3   | 0   | 61  | 3   | 0   | 58  | -  | .3 |
|                       | .97 | .18 | .03 | .90 | .44 | .95 | 0. | 89 |
|                       |     |     |     |     |     |     | 86 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Word wall           | 3   | 0   | 62  | 3   | 0   | 57  | -  | .1 |
|                       | .95 | .22 | .08 | .83 | .53 | .89 | 1. | 71 |
|                       |     |     |     |     |     |     | 37 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Classroom library   | 3   | 0   | 60  | 3   | 0   | 59  | -  | .9 |
|                       | .97 | .18 | .03 | .93 | .41 | .97 | 0. | 80 |
|                       |     |     |     |     |     |     | 03 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Evidence of student | 3   | 1   | 65  | 2   | 1   | 53  | -  | .0 |
| > writing/work        | .20 | .07 | .96 | .76 | .25 | .94 | 2. | 42 |
| > products            |     |     |     |     |     |     | 03 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > ***MATERIALS        |     |     |     |     |     |     |    |    |
| > USED***             |     |     |     |     |     |     |    |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Basal texts         | 2   | 0   | 68  | 1   | 1   | 51  | -  | .0 |
|                       | .08 | .85 | .23 | .56 | .02 | .64 | 2. | 06 |
|                       |     |     |     |     |     |     | 74 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Big books           | 0   | 0   | 63  | 0   | 0   | 55  | -  | .1 |
|                       | .58 | .62 | .31 | .41 | .50 | .69 | 1. | 67 |
|                       |     |     |     |     |     |     | 38 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Books on tape       | 0   | 0   | 64  | 0   | 0   | 55  | -  | .0 |
|                       | .55 | .67 | .50 | .34 | .51 | .42 | 1. | 95 |
|                       |     |     |     |     |     |     | 67 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Computers           | 1   | 1   | 66  | 0   | 0   | 53  | -  | .0 |
|                       | .20 | .04 | .14 | .80 | .74 | .75 | 2. | 39 |
|                       |     |     |     |     |     |     | 06 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Fiction books       | 1   | 1   | 60  | 1   | 1   | 59  | -  | .7 |
|                       | .12 | .11 | .97 | .03 | .02 | .02 | 0. | 47 |
|                       |     |     |     |     |     |     | 32 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Non-fiction books   | 0   | 0   | 65  | 0   | 0   | 54  | -  | .0 |
|                       | .47 | .70 | .47 | .19 | .39 | .44 | 2. | 25 |
|                       |     |     |     |     |     |     | 25 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Poetry              | 0   | 0   | 60  | 0   | 0   | 58  | -  | .5 |
|                       | .25 | .44 | .87 | .22 | .46 | .13 | 0. | 52 |
|                       |     |     |     |     |     |     | 59 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Newspaper/magazines | 0   | 0   | 58  | 0   | 0   | 61  | 0. | .4 |
|                       | .07 | .25 | .97 | .10 | .30 | .05 | 68 | 96 |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Word/vocabulary     | 2   | 0   | 69  | 1   | 1   | 50  | -  | .0 |
| > materials           | .07 | .99 | .53 | .51 | .01 | .31 | 3. | 02 |
|                       |     |     |     |     |     |     | 17 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| >                     | 1   | 0   | 63  | 1   | 0   | 56  | -  | .2 |
|  Worksheets/workbooks | .67 | .95 | .41 | .46 | .70 | .53 | 1. | 44 |
|                       |     |     |     |     |     |     | 16 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Student             | 0   | 0   | 60  | 0   | 0   | 57  | -  | .5 |
| > notebooks/journals  | .58 | .77 | .64 | .47 | .63 | .27 | 0. | 43 |
|                       |     |     |     |     |     |     | 61 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+
| > Chart               | 1   | 1   | 64  | 1   | 1   | 55  | -  | .1 |
| > tablets/whiteboards | .92 | .09 | .84 | .63 | .02 | .07 | 1. | 09 |
|                       |     |     |     |     |     |     | 60 |    |
+-----------------------+-----+-----+-----+-----+-----+-----+----+----+

> *\*P\<.001*

**TEACHER QUESTIONNAIRES (RFTQ AND RFTQ-C)**

***Descriptive Results***

The results of the questionnaires administered in the spring of 2007 are
summarized in Table 6. Overall, teachers in both Reading First and
control schools were positive concerning their schools, professional
development, and literacy programs. However, for every item, a larger
majority of Reading First teachers responded positively than their
control counterparts. In fact, of the 27 items listed on the
questionnaire, more than 80% of the Reading First teachers were in
agreement with 21 of the items; for control schools, there were 10 items
in which more than 80% of the teachers agreed. Areas in which there were
the greatest disparities between the two groups included:

-   the impact the literacy program had on instruction (95.0% for
    Reading First teachers vs. 61.4% for control; item 14);

-   the belief that all children would read at grade level or above by
    third grade ( 87.4% for Reading First teachers vs. 56.6% for
    control; item 26)

-   the amount of individual feedback and coaching they received (86.6%
    treatment vs. 58.2% control; item 24);

-   the degree of children's enthusiasm for reading due to the literacy
    program ( 79.4% for Reading First teachers vs. 55.6% for control
    teachers; item 16)

-   the availability of scientifically-based materials (97.7% for RF
    teachers vs. 77.2% for control; item 11);

-   the degree of support from Literacy Leaders (92.7% treatment vs.
    72.0 % control; item 5); and,

-   the positive impact on student achievement (93.5% for Reading First
    teachers vs. 74.1% for control teachers; item 15).

Items that were less positive were consistently so between the two
groups. Slightly more than sixty percent (61.5%) of the Reading First
teachers agreed that they were given enough planning time for
implementation; 49.2% of control teachers agreed with the statement.
Regarding parental involvement, 56.5% of the Reading First teachers
agreed that parents were more involved due to Reading First; for control
teachers, 37.0% agreed their literacy program had increased parental
involvement. Finally, just over half (50.8%) of the Reading First
teachers agreed that student misbehavior did not interfere with reading
instruction, for control teachers only 30.7% responded positively.

### Demographic data

Reading First teachers participating in the survey appeared to be
slightly younger than control respondents; 51.5% of treatment teachers
were 39 years or less; whereas only 43.4% of control teachers reported
their ages to be less than forty years. There was an approximate
difference of 5 percentage points in the 40-49 age group, with 22.2% of
the control school respondents reporting their ages in this category;
for Reading First teachers, 17.6% fell into this category. Other age
categories had only minor differences.

In terms of ethnicity, both groups were comparable. Treatment teachers
responding to the survey were 63.7% White and 34.0% African American;
while teachers in control schools were 67.2% White, and 27% African
American. In addition, the vast majority of the respondents in both
groups were female.

Educational level was also comparable between groups. Of the treatment
teachers, 42.0% reported having a bachelor's degree; for the control
group respondents, this increased slightly to 43.4%. Half of the
treatment teachers (50.0%) reported attaining a Master's degree; for
control teachers, 42.3% reported a Master's degree. However, 12.2% of
the control group reported a degree beyond a Master's--- 7.6% of the
treatment teachers responding reported a degree beyond a Master's.
Regarding teaching experience, teachers from both groups responded
similarly. Control teachers were slightly more likely to have more than
20 years experience (27.0%) than their Reading First peers (20.2%).
Demographic information is summarized in Table 7.

### Open-ended responses

Comments provided by the Reading First teachers were mostly positive,
and many commended their school's literacy leaders for offering the
necessary support and direction to enable effective implementation of
the program. Lack of planning time and increased paperwork were listed
as negative factor for some of the Reading First teachers. Control
teachers provided fewer comments, with the majority of their responses
either neutral or negative. Concerns listed included lack of a cohesive
literacy program, increased professional development opportunities, and
more materials. The open-ended responses to the survey are listed in
Tables 8 and 9.

**Table 6: Reading First Teacher Questionnaire (RFTQ):**

**Treatment Schools (T) N = 262**

**Control Schools (C) N = 189**

+---+--------------------------------------+---+---+---+---+----+---+---+
| # |                                      | * |   |   |   |    |   |   |
| # |                                      | * |   |   |   |    |   |   |
| # |                                      | P |   |   |   |    |   |   |
| # |                                      | e |   |   |   |    |   |   |
| # |                                      | r |   |   |   |    |   |   |
| # |                                      | c |   |   |   |    |   |   |
| # |                                      | e |   |   |   |    |   |   |
| # |                                      | n |   |   |   |    |   |   |
| # |                                      | t |   |   |   |    |   |   |
|   |                                      | o |   |   |   |    |   |   |
| S |                                      | f |   |   |   |    |   |   |
| e |                                      | T |   |   |   |    |   |   |
| c |                                      | e |   |   |   |    |   |   |
| t |                                      | a |   |   |   |    |   |   |
| i |                                      | c |   |   |   |    |   |   |
| o |                                      | h |   |   |   |    |   |   |
| n |                                      | e |   |   |   |    |   |   |
|   |                                      | r |   |   |   |    |   |   |
| 1 |                                      | R |   |   |   |    |   |   |
|   |                                      | e |   |   |   |    |   |   |
| * |                                      | s |   |   |   |    |   |   |
| * |                                      | p |   |   |   |    |   |   |
| S |                                      | o |   |   |   |    |   |   |
| u |                                      | n |   |   |   |    |   |   |
| m |                                      | s |   |   |   |    |   |   |
| m |                                      | e |   |   |   |    |   |   |
| a |                                      | * |   |   |   |    |   |   |
| r |                                      | * |   |   |   |    |   |   |
| y |                                      |   |   |   |   |    |   |   |
| o |                                      |   |   |   |   |    |   |   |
| f |                                      |   |   |   |   |    |   |   |
| T |                                      |   |   |   |   |    |   |   |
| e |                                      |   |   |   |   |    |   |   |
| a |                                      |   |   |   |   |    |   |   |
| c |                                      |   |   |   |   |    |   |   |
| h |                                      |   |   |   |   |    |   |   |
| e |                                      |   |   |   |   |    |   |   |
| r |                                      |   |   |   |   |    |   |   |
| P |                                      |   |   |   |   |    |   |   |
| e |                                      |   |   |   |   |    |   |   |
| r |                                      |   |   |   |   |    |   |   |
| c |                                      |   |   |   |   |    |   |   |
| e |                                      |   |   |   |   |    |   |   |
| p |                                      |   |   |   |   |    |   |   |
| t |                                      |   |   |   |   |    |   |   |
| i |                                      |   |   |   |   |    |   |   |
| o |                                      |   |   |   |   |    |   |   |
| n |                                      |   |   |   |   |    |   |   |
| s |                                      |   |   |   |   |    |   |   |
| * |                                      |   |   |   |   |    |   |   |
| * |                                      |   |   |   |   |    |   |   |
+---+--------------------------------------+---+---+---+---+----+---+---+
|   |                                      | * |   |   | * |    | * |   |
|   |                                      | * |   |   | * |    | * |   |
|   |                                      | D |   |   | N |    | S |   |
|   |                                      | i |   |   | e |    | t |   |
|   |                                      | s |   |   | u |    | r |   |
|   |                                      | a |   |   | t |    | o |   |
|   |                                      | g |   |   | r |    | n |   |
|   |                                      | r |   |   | a |    | g |   |
|   |                                      | e |   |   | l |    | l |   |
|   |                                      | e |   |   | * |    | y |   |
|   |                                      | & |   |   | * |    | A |   |
|   |                                      | S |   |   |   |    | g |   |
|   |                                      | t |   |   |   |    | r |   |
|   |                                      | r |   |   |   |    | e |   |
|   |                                      | o |   |   |   |    | e |   |
|   |                                      | n |   |   |   |    | & |   |
|   |                                      | g |   |   |   |    | A |   |
|   |                                      | l |   |   |   |    | g |   |
|   |                                      | y |   |   |   |    | r |   |
|   |                                      | D |   |   |   |    | e |   |
|   |                                      | i |   |   |   |    | e |   |
|   |                                      | s |   |   |   |    | * |   |
|   |                                      | a |   |   |   |    | * |   |
|   |                                      | g |   |   |   |    |   |   |
|   |                                      | r |   |   |   |    |   |   |
|   |                                      | e |   |   |   |    |   |   |
|   |                                      | e |   |   |   |    |   |   |
|   |                                      | * |   |   |   |    |   |   |
|   |                                      | * |   |   |   |    |   |   |
+---+--------------------------------------+---+---+---+---+----+---+---+
|   |                                      | * | * | * |   | *  | * | * |
|   |                                      | * | * | * |   | *C | * | * |
|   |                                      | T | C | T |   | ** | T | C |
|   |                                      | * | * | * |   |    | * | * |
|   |                                      | * | * | * |   |    | * | * |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | I have a thorough understanding of   | 1 | 1 | 1 |   | 9  | 9 | 8 |
|   | this school\'s RF/literacy program.  | . | . | . |   | .5 | 6 | 8 |
|   |                                      | 9 | 1 | 5 |   |    | . | . |
|   |                                      |   |   |   |   |    | 6 | 4 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 2 | I have received adequate initial and | 3 | 6 | 1 |   | 11 | 9 | 8 |
|   | ongoing professional development/    | . | . | 2 |   | .5 | 6 | 3 |
|   | training for implementation of my    | 7 | 6 | . |   |    | . | . |
|   | school\'s RF/literacy program.       |   |   | 2 |   |    | 1 | 1 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 3 | Professional development provided by | 1 | 2 | 3 |   | 6  | 9 | 8 |
|   | the Tennessee Reading Cadre/ my      | . | . | . |   | .3 | 5 | 9 |
|   | school has been valuable.            | 1 | 6 | 8 |   |    | . | . |
|   |                                      |   |   |   |   |    | 0 | 9 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 4 | Professional development provided by | 1 | 2 | 8 |   | 12 | 8 | 8 |
|   | the Tennessee Department of          | . | . | . |   | .2 | 9 | 3 |
|   | Education/external trainers has been | 9 | 1 | 0 |   |    | . | . |
|   | valuable.                            |   |   |   |   |    | 7 | 6 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 5 | Guidance and support provided by our | 1 | 9 | 5 |   | 15 | 9 | 7 |
|   | school\'s Literacy Leader have       | . | . | . |   | .9 | 2 | 2 |
|   | helped our school implement its RF   | 5 | 0 | 3 |   |    | . | . |
|   | program.                             |   |   |   |   |    | 7 | 0 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 6 | The principal is an effective        | 1 | 1 | 6 |   | 12 | 9 | 8 |
|   | instructional leader.                | . | . | . |   | .7 | 2 | 4 |
|   |                                      | 5 | 6 | 5 |   |    | . | . |
|   |                                      |   |   |   |   |    | 0 | 7 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 7 | Teachers are given sufficient        | 2 | 3 | 1 |   | 18 | 6 | 4 |
|   | planning time to implement our       | 4 | 1 | 3 |   | .0 | 1 | 9 |
|   | RF/literacy program.                 | . | . | . |   |    | . | . |
|   |                                      | 4 | 2 | 7 |   |    | 5 | 2 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 8 | All instructional elements of        | 0 | 3 | 1 |   | 8  | 9 | 8 |
|   | RF/literacy at our school -          | . | . | . |   | .5 | 8 | 7 |
|   | assessments, programs, supplemental  | 0 | 7 | 1 |   |    | . | . |
|   | materials -- are based on            |   |   |   |   |    | 9 | 8 |
|   | scientifically-based reading         |   |   |   |   |    |   |   |
|   | research.                            |   |   |   |   |    |   |   |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 9 | Because of RF/our literacy program,  | 0 | 2 | 1 |   | 14 | 9 | 8 |
|   | systematic and explicit instruction  | . | . | . |   | .3 | 7 | 3 |
|   | in phonemic awareness, phonics,      | 4 | 1 | 9 |   |    | . | . |
|   | vocabulary, fluency, and             |   |   |   |   |    | 7 | 6 |
|   | comprehension occurs daily in our    |   |   |   |   |    |   |   |
|   | school's K-3 classrooms.             |   |   |   |   |    |   |   |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | Because of our RF/our literacy       | 0 | 1 | 1 |   | 10 | 9 | 8 |
| 0 | program, teachers use assessments to | 4 | . | . |   | .6 | 8 | 7 |
|   | inform instructional decisions and   |   | 1 | 5 |   |    | . | . |
|   | monitor student achievement.         |   |   |   |   |    | 1 | 8 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | Scientifically-based instructional   | 1 | 5 | 1 |   | 16 | 9 | 7 |
| 1 | materials (books, assessments, and   | . | . | . |   | .9 | 7 | 7 |
|   | other resources) needed to implement | 1 | 3 | 1 |   |    | . | . |
|   | our RF/literacy program are readily  |   |   |   |   |    | 7 | 2 |
|   | available.                           |   |   |   |   |    |   |   |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | Our school has sufficient faculty    | 1 | 1 | 6 |   | 12 | 7 | 6 |
| 2 | and staff to fully implement its     | 4 | 5 | . |   | .7 | 8 | 8 |
|   | RF/literacy program.                 | . | . | 9 |   |    | . | . |
|   |                                      | 9 | 9 |   |   |    | 2 | 8 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | The administration protects a daily  | 3 | 1 | 3 |   | 10 | 9 | 7 |
| 3 | uninterrupted 90 or 120 minute block | . | 0 | . |   | .6 | 3 | 7 |
|   | of reading instruction.              | 1 | . | 8 |   |    | . | . |
|   |                                      |   | 1 |   |   |    | 1 | 8 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | Our RF/literacy program has changed  | 0 | 9 | 4 |   | 28 | 9 | 6 |
| 4 | my reading instruction activities a  | . | . | . |   | .0 | 5 | 1 |
|   | great deal.                          | 8 | 0 | 2 |   |    | . | . |
|   |                                      |   |   |   |   |    | 0 | 4 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | Student achievement has been         | 0 | 1 | 5 |   | 21 | 9 | 7 |
| 5 | positively impacted by our RF/       | . | . | . |   | .2 | 3 | 4 |
|   | literacy program.                    | 8 | 6 | 3 |   |    | . | . |
|   |                                      |   |   |   |   |    | 5 | 1 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | Children in this school are more     | 5 | 8 | 1 |   | 32 | 7 | 5 |
| 6 | enthusiastic about reading and       | . | . | 4 |   | .3 | 9 | 5 |
|   | learning because of our RF/literacy  | 7 | 5 | . |   |    | . | . |
|   | program.                             |   |   | 5 |   |    | 4 | 6 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | Because of our RF/literacy program,  | 1 | 2 | 2 |   | 33 | 5 | 3 |
| 7 | parents are more involved in the     | 7 | 6 | 5 |   | .9 | 6 | 7 |
|   | literacy program of this school.     | . | . | . |   |    | . | . |
|   |                                      | 9 | 5 | 6 |   |    | 5 | 0 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | Our school is effectively            | 0 | 2 | 3 |   | 15 | 9 | 7 |
| 8 | implementing the 3-Tier Reading      | . | . | . |   | .3 | 5 | 9 |
|   | model/it literacy program.           | 4 | 6 | 4 |   |    | . | . |
|   |                                      |   |   |   |   |    | 8 | 4 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 1 | Our RF/literacy program adequately   | 8 | 1 | 1 |   | 25 | 7 | 6 |
| 9 | addresses the requirements of        | . | 2 | 2 |   | .9 | 9 | 0 |
|   | children with special needs.         | 0 | . | . |   |    | . | . |
|   |                                      |   | 2 | 6 |   |    | 0 | 8 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 2 | Because of our RF/literacy program,  | 5 | 1 | 1 |   | 21 | 8 | 6 |
| 0 | teachers in this school spend more   | . | 2 | 1 |   | .2 | 3 | 5 |
|   | time working together to develop     | 0 | . | . |   |    | . | . |
|   | curriculum and plan instruction.     |   | 2 | 8 |   |    | 2 | 6 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 2 | Teachers in this school are          | 1 | 2 | 1 |   | 14 | 8 | 8 |
| 1 | generally supportive of our          | . | . | 0 |   | .3 | 8 | 3 |
|   | RF/literacy program.                 | 5 | 1 | . |   |    | . | . |
|   |                                      |   |   | 3 |   |    | 2 | 1 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 2 | The elements of our RF/literacy      | 1 | 1 | 2 |   | 14 | 9 | 8 |
| 2 | program are effectively integrated   | . | . | . |   | .8 | 6 | 3 |
|   | to help us meet school improvement   | 1 | 1 | 7 |   |    | . | . |
|   | goals.                               |   |   |   |   |    | 2 | 1 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 2 | This school has a plan for           | 1 | 4 | 6 |   | 18 | 9 | 7 |
| 3 | evaluating all components of our     | . | . | . |   | .5 | 2 | 3 |
|   | RF/literacy program.                 | 1 | 2 | 1 |   |    | . | . |
|   |                                      |   |   |   |   |    | 4 | 5 |
+---+--------------------------------------+---+---+---+---+----+---+---+

**Table 6: Continued**

+---+--------------------------------------+---+---+---+---+----+---+---+
| # |                                      | * |   |   |   |    |   |   |
| # |                                      | * |   |   |   |    |   |   |
| # |                                      | P |   |   |   |    |   |   |
| # |                                      | e |   |   |   |    |   |   |
| # |                                      | r |   |   |   |    |   |   |
| # |                                      | c |   |   |   |    |   |   |
| # |                                      | e |   |   |   |    |   |   |
| # |                                      | n |   |   |   |    |   |   |
| # |                                      | t |   |   |   |    |   |   |
|   |                                      | o |   |   |   |    |   |   |
| S |                                      | f |   |   |   |    |   |   |
| e |                                      | T |   |   |   |    |   |   |
| c |                                      | e |   |   |   |    |   |   |
| t |                                      | a |   |   |   |    |   |   |
| i |                                      | c |   |   |   |    |   |   |
| o |                                      | h |   |   |   |    |   |   |
| n |                                      | e |   |   |   |    |   |   |
|   |                                      | r |   |   |   |    |   |   |
| 1 |                                      | R |   |   |   |    |   |   |
|   |                                      | e |   |   |   |    |   |   |
| c |                                      | s |   |   |   |    |   |   |
| o |                                      | p |   |   |   |    |   |   |
| n |                                      | o |   |   |   |    |   |   |
| t |                                      | n |   |   |   |    |   |   |
| i |                                      | s |   |   |   |    |   |   |
| n |                                      | e |   |   |   |    |   |   |
| u |                                      | * |   |   |   |    |   |   |
| e |                                      | * |   |   |   |    |   |   |
| d |                                      |   |   |   |   |    |   |   |
|   |                                      |   |   |   |   |    |   |   |
| * |                                      |   |   |   |   |    |   |   |
| * |                                      |   |   |   |   |    |   |   |
| S |                                      |   |   |   |   |    |   |   |
| u |                                      |   |   |   |   |    |   |   |
| m |                                      |   |   |   |   |    |   |   |
| m |                                      |   |   |   |   |    |   |   |
| a |                                      |   |   |   |   |    |   |   |
| r |                                      |   |   |   |   |    |   |   |
| y |                                      |   |   |   |   |    |   |   |
| o |                                      |   |   |   |   |    |   |   |
| f |                                      |   |   |   |   |    |   |   |
| T |                                      |   |   |   |   |    |   |   |
| e |                                      |   |   |   |   |    |   |   |
| a |                                      |   |   |   |   |    |   |   |
| c |                                      |   |   |   |   |    |   |   |
| h |                                      |   |   |   |   |    |   |   |
| e |                                      |   |   |   |   |    |   |   |
| r |                                      |   |   |   |   |    |   |   |
| P |                                      |   |   |   |   |    |   |   |
| e |                                      |   |   |   |   |    |   |   |
| r |                                      |   |   |   |   |    |   |   |
| c |                                      |   |   |   |   |    |   |   |
| e |                                      |   |   |   |   |    |   |   |
| p |                                      |   |   |   |   |    |   |   |
| t |                                      |   |   |   |   |    |   |   |
| i |                                      |   |   |   |   |    |   |   |
| o |                                      |   |   |   |   |    |   |   |
| n |                                      |   |   |   |   |    |   |   |
| s |                                      |   |   |   |   |    |   |   |
| * |                                      |   |   |   |   |    |   |   |
| * |                                      |   |   |   |   |    |   |   |
+---+--------------------------------------+---+---+---+---+----+---+---+
|   |                                      | * |   |   | * |    | * |   |
|   |                                      | * |   |   | * |    | * |   |
|   |                                      | D |   |   | N |    | S |   |
|   |                                      | i |   |   | e |    | t |   |
|   |                                      | s |   |   | u |    | r |   |
|   |                                      | a |   |   | t |    | o |   |
|   |                                      | g |   |   | r |    | n |   |
|   |                                      | r |   |   | a |    | g |   |
|   |                                      | e |   |   | l |    | l |   |
|   |                                      | e |   |   | * |    | y |   |
|   |                                      | & |   |   | * |    | A |   |
|   |                                      | S |   |   |   |    | g |   |
|   |                                      | t |   |   |   |    | r |   |
|   |                                      | r |   |   |   |    | e |   |
|   |                                      | o |   |   |   |    | e |   |
|   |                                      | n |   |   |   |    | & |   |
|   |                                      | g |   |   |   |    | A |   |
|   |                                      | l |   |   |   |    | g |   |
|   |                                      | y |   |   |   |    | r |   |
|   |                                      | D |   |   |   |    | e |   |
|   |                                      | i |   |   |   |    | e |   |
|   |                                      | s |   |   |   |    | * |   |
|   |                                      | a |   |   |   |    | * |   |
|   |                                      | g |   |   |   |    |   |   |
|   |                                      | r |   |   |   |    |   |   |
|   |                                      | e |   |   |   |    |   |   |
|   |                                      | e |   |   |   |    |   |   |
|   |                                      | * |   |   |   |    |   |   |
|   |                                      | * |   |   |   |    |   |   |
+---+--------------------------------------+---+---+---+---+----+---+---+
|   |                                      | * | * | * |   | *  | * | * |
|   |                                      | * | * | * |   | *C | * | * |
|   |                                      | T | C | T |   | ** | T | C |
|   |                                      | * | * | * |   |    | * | * |
|   |                                      | * | * | * |   |    | * | * |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 2 | Because of RF/ our literacy program, | 3 | 1 | 9 |   | 28 | 8 | 5 |
| 4 | I have received more individual      | . | 2 | . |   | .6 | 6 | 8 |
|   | feedback and coaching related to my  | 8 | . | 5 |   |    | . | . |
|   | reading instruction activities.      |   | 2 |   |   |    | 6 | 2 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 2 | Teachers are encouraged to           | 5 | 3 | 7 |   | 21 | 8 | 7 |
| 5 | communicate concerns, questions, and | . | . | . |   | .2 | 7 | 4 |
|   | constructive ideas regarding the     | 0 | 7 | 3 |   |    | . | . |
|   | RF/literacy program.                 |   |   |   |   |    | 8 | 6 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 2 | The faculty, staff, and              | 3 | 2 | 8 |   | 22 | 8 | 5 |
| 6 | administration believe that all      | . | 0 | . |   | .8 | 7 | 6 |
|   | children can read at grade level or  | 8 | . | 4 |   |    | . | . |
|   | above by the end of the third grade. |   | 1 |   |   |    | 4 | 6 |
+---+--------------------------------------+---+---+---+---+----+---+---+
| 2 | Student misbehavior in this school   | 2 | 4 | 1 |   | 20 | 5 | 3 |
| 7 | does not interfere with the teaching | 9 | 7 | 9 |   | .1 | 0 | 0 |
|   | of reading.                          | . | . | . |   |    | . | . |
|   |                                      | 8 | 6 | 1 |   |    | 8 | 7 |
+---+--------------------------------------+---+---+---+---+----+---+---+

\*Note: Item percentages may not total 100% because of missing input
from some respondents.

**Table 7: Reading First Teacher Questionnaire (RFTQ): Year 2
(2005-2006)**

**Respondent Demographics**

**Treatment Schools (T) N = 262**

**Control Schools (C) N = 189**

+---+----------------------------------+---------------+---------------+
| # |                                  | **Percent of  |               |
| # |                                  | Teacher       |               |
| # |                                  | Response**    |               |
| # |                                  |               |               |
| # |                                  |               |               |
| # |                                  |               |               |
| # |                                  |               |               |
| # |                                  |               |               |
| # |                                  |               |               |
|   |                                  |               |               |
| S |                                  |               |               |
| e |                                  |               |               |
| c |                                  |               |               |
| t |                                  |               |               |
| i |                                  |               |               |
| o |                                  |               |               |
| n |                                  |               |               |
|   |                                  |               |               |
| 2 |                                  |               |               |
|   |                                  |               |               |
| * |                                  |               |               |
| * |                                  |               |               |
| R |                                  |               |               |
| e |                                  |               |               |
| s |                                  |               |               |
| p |                                  |               |               |
| o |                                  |               |               |
| n |                                  |               |               |
| d |                                  |               |               |
| e |                                  |               |               |
| n |                                  |               |               |
| t |                                  |               |               |
| D |                                  |               |               |
| e |                                  |               |               |
| m |                                  |               |               |
| o |                                  |               |               |
| g |                                  |               |               |
| r |                                  |               |               |
| a |                                  |               |               |
| p |                                  |               |               |
| h |                                  |               |               |
| i |                                  |               |               |
| c |                                  |               |               |
| s |                                  |               |               |
| * |                                  |               |               |
| * |                                  |               |               |
+---+----------------------------------+---------------+---------------+
|   |                                  | **T**         | **C**         |
+---+----------------------------------+---------------+---------------+
| T |                                  |               |               |
| o |                                  |               |               |
| t |                                  |               |               |
| a |                                  |               |               |
| l |                                  |               |               |
| Y |                                  |               |               |
| e |                                  |               |               |
| a |                                  |               |               |
| r |                                  |               |               |
| s |                                  |               |               |
| o |                                  |               |               |
| f |                                  |               |               |
| E |                                  |               |               |
| x |                                  |               |               |
| p |                                  |               |               |
| e |                                  |               |               |
| r |                                  |               |               |
| i |                                  |               |               |
| e |                                  |               |               |
| n |                                  |               |               |
| c |                                  |               |               |
| e |                                  |               |               |
| i |                                  |               |               |
| n |                                  |               |               |
| a |                                  |               |               |
| n |                                  |               |               |
| y |                                  |               |               |
| S |                                  |               |               |
| c |                                  |               |               |
| h |                                  |               |               |
| o |                                  |               |               |
| o |                                  |               |               |
| l |                                  |               |               |
+---+----------------------------------+---------------+---------------+
|   | 5 years or less                  | 25.6          | 23.3          |
+---+----------------------------------+---------------+---------------+
|   | 6 -- 10 years                    | 29.4          | 24.9          |
+---+----------------------------------+---------------+---------------+
|   | 11 -- 15 years                   | 14.5          | 13.8          |
+---+----------------------------------+---------------+---------------+
|   | 16 -- 20 years                   | 10.3          | 10.6          |
+---+----------------------------------+---------------+---------------+
|   | More than 20 years               | 20.2          | 27.0          |
+---+----------------------------------+---------------+---------------+
| T |                                  |               |               |
| o |                                  |               |               |
| t |                                  |               |               |
| a |                                  |               |               |
| l |                                  |               |               |
| Y |                                  |               |               |
| e |                                  |               |               |
| a |                                  |               |               |
| r |                                  |               |               |
| s |                                  |               |               |
| o |                                  |               |               |
| f |                                  |               |               |
| E |                                  |               |               |
| x |                                  |               |               |
| p |                                  |               |               |
| e |                                  |               |               |
| r |                                  |               |               |
| i |                                  |               |               |
| e |                                  |               |               |
| n |                                  |               |               |
| c |                                  |               |               |
| e |                                  |               |               |
| i |                                  |               |               |
| n |                                  |               |               |
| t |                                  |               |               |
| h |                                  |               |               |
| i |                                  |               |               |
| s |                                  |               |               |
| S |                                  |               |               |
| c |                                  |               |               |
| h |                                  |               |               |
| o |                                  |               |               |
| o |                                  |               |               |
| l |                                  |               |               |
+---+----------------------------------+---------------+---------------+
|   | Less than 1 year                 | 10.7          | 12.7          |
+---+----------------------------------+---------------+---------------+
|   | 1 - 5 years                      | 43.5          | 39.2          |
+---+----------------------------------+---------------+---------------+
|   | 6 - 10 years                     | 26.3          | 25.4          |
+---+----------------------------------+---------------+---------------+
|   | 11 - 15 years                    | 8.0           | 9.0           |
+---+----------------------------------+---------------+---------------+
|   | More than 15 years               | 11.5          | 13.2          |
+---+----------------------------------+---------------+---------------+
| E |                                  |               |               |
| d |                                  |               |               |
| u |                                  |               |               |
| c |                                  |               |               |
| a |                                  |               |               |
| t |                                  |               |               |
| i |                                  |               |               |
| o |                                  |               |               |
| n |                                  |               |               |
| a |                                  |               |               |
| l |                                  |               |               |
| A |                                  |               |               |
| t |                                  |               |               |
| t |                                  |               |               |
| a |                                  |               |               |
| i |                                  |               |               |
| n |                                  |               |               |
| m |                                  |               |               |
| e |                                  |               |               |
| n |                                  |               |               |
| t |                                  |               |               |
+---+----------------------------------+---------------+---------------+
|   | High School Diploma or less      | 0.4           | 0.0           |
+---+----------------------------------+---------------+---------------+
|   | Associate's Degree or some       | 0.0           | 0.0           |
|   | college credit                   |               |               |
+---+----------------------------------+---------------+---------------+
|   | Bachelor's Degree                | 42.0          | 43.4          |
+---+----------------------------------+---------------+---------------+
|   | Master's Degree                  | 50.0          | 42.3          |
+---+----------------------------------+---------------+---------------+
|   | Degree beyond Master's           | 7.6           | 12.2          |
+---+----------------------------------+---------------+---------------+
| E |                                  |               |               |
| t |                                  |               |               |
| h |                                  |               |               |
| n |                                  |               |               |
| i |                                  |               |               |
| c |                                  |               |               |
| i |                                  |               |               |
| t |                                  |               |               |
| y |                                  |               |               |
| / |                                  |               |               |
| R |                                  |               |               |
| a |                                  |               |               |
| c |                                  |               |               |
| e |                                  |               |               |
+---+----------------------------------+---------------+---------------+
|   | American Indian or Alaskan       | 0.0           | 0.5           |
|   | Native                           |               |               |
+---+----------------------------------+---------------+---------------+
|   | Asian or Pacific Islander        | 0.0           | 1.1           |
+---+----------------------------------+---------------+---------------+
|   | Black / African American         | 34.0          | 27.0          |
+---+----------------------------------+---------------+---------------+
|   | Hispanic / Latino                | 0.4           | 0.0           |
+---+----------------------------------+---------------+---------------+
|   | Multi-Racial                     | 0.4           | 1.6           |
+---+----------------------------------+---------------+---------------+
|   | White                            | 63.7          | 67.2          |
+---+----------------------------------+---------------+---------------+
| A |                                  |               |               |
| g |                                  |               |               |
| e |                                  |               |               |
| G |                                  |               |               |
| r |                                  |               |               |
| o |                                  |               |               |
| u |                                  |               |               |
| p |                                  |               |               |
+---+----------------------------------+---------------+---------------+
|   | 29 years or less                 | 22.1          | 16.9          |
+---+----------------------------------+---------------+---------------+
|   | 30 - 39 years                    | 29.4          | 26.5          |
+---+----------------------------------+---------------+---------------+
|   | 40 - 49 years                    | 17.6          | 22.2          |
+---+----------------------------------+---------------+---------------+
|   | 50 - 59 years                    | 26.0          | 27.0          |
+---+----------------------------------+---------------+---------------+
|   | 60 years or older                | 4.6           | 2.1           |
+---+----------------------------------+---------------+---------------+
| G |                                  |               |               |
| e |                                  |               |               |
| n |                                  |               |               |
| d |                                  |               |               |
| e |                                  |               |               |
| r |                                  |               |               |
+---+----------------------------------+---------------+---------------+
|   | Male                             | 1.9           | 1.6           |
+---+----------------------------------+---------------+---------------+
|   | Female                           | 96.9          | 95.2          |
+---+----------------------------------+---------------+---------------+

Note: Percentages may not total 100% because of missing input from some
respondents.

**Table 8: Reading First Open-Ended Teacher Responses on the RFTQ**

+-----------------------------------------------------------------------+
| ***[Positive Comments]{.underline}***                                 |
|                                                                       |
| **Please provide any additional comments you may have pertaining to   |
| your school's literacy program.**                                     |
+-----------------------------------------------------------------------+
| *[Responses]{.underline}*                                             |
+-----------------------------------------------------------------------+
| 1.  Our literacy leader does a really good job. She has high          |
|     expectations but is very understanding. She strives to help us in |
|     all areas of teaching reading. She is very helpful! Discipline    |
|     greatly affects the success of the reading program as does        |
|     teacher moral.                                                    |
|                                                                       |
| 2.  Every fiber of our K-2 literacy program has been marinated by way |
|     of the RF program. We have such a variety of books to meet the    |
|     needs of every child We\'re so fortunate to have the funding of   |
|     this program for our students. THANK YOU.                         |
|                                                                       |
| 3.  Our school works incredibly hard to meet the reading needs of our |
|     students.                                                         |
|                                                                       |
| 4.  I feel that our school has more up to date materials than it has  |
|     had in 30 years!! Thank God for Reading First which gives our     |
|     economically disadvantaged area the same benefits as other        |
|     schools.                                                          |
|                                                                       |
| 5.  The RF program has allowed us to overcome obstacles to providing  |
|     specialized instruction to all of our students.                   |
|                                                                       |
| 6.  Our staff is awesome! We all want all students to learn and be    |
|     successful!                                                       |
|                                                                       |
| 7.  RF has been a wonderful thing for our school and our children.    |
|                                                                       |
| 8.  Because of RF at \[name\], I feel well equip to incorporate the 5 |
|     components with the resources that are provided\...I have seen a  |
|     growth in my students with learning disabilities\...I feel        |
|     motivated by the Literacy Leader as well as our principal\...I    |
|     feel like I am a part of a team who strives for excellence in all |
|     students as well as for the teacher. (sic)                        |
|                                                                       |
| 9.  Because of RF I feel more adequate in teaching my students with   |
|     learning disabilities because of the various resources,           |
|     professional developments, and support                            |
|                                                                       |
| 10. Good program!                                                     |
|                                                                       |
| 11. Wonderful Literacy Leader                                         |
|                                                                       |
| 12. I believe with the continuous implementation of the Reading First |
|     Program, our students at \[name\] will truly be reading by third  |
|     grade.                                                            |
|                                                                       |
| 13. I have found this program to be a great benefit for our students. |
|     Students that have participated in the program since Kindergarten |
|     are more skilled at letter recognition, decoding, blending        |
|     sounds, and using what they know to detect means of unfamiliar    |
|     words using context clues. Reading First, when used properly is a |
|     strong foundation for beginning and developing readers.           |
|                                                                       |
| 14. Our Literacy Leader works tirelessly to accommodate teachers. She |
|     goes beyond the call of duty to support classroom instruction.    |
|     She deserves a RF crown.                                          |
|                                                                       |
| 15. Our school has had a very good cadre leader and a superb Reading  |
|     First Leader who have provided us with valuable information and   |
|     activities to use in our classrooms as well as valuable feedback  |
|     to help us make improvements where needed.                        |
|                                                                       |
| 16. Reading First has proven to have positive results for the         |
|     students at our school. Students that have been with the program  |
|     for two years or more have a definite advantage over the other    |
|     students.                                                         |
|                                                                       |
| 17. Reading First runs smoothly at our school. I must give credit to  |
|     our Principal and Literacy Leader. Both are to be commended for   |
|     their diligence.                                                  |
|                                                                       |
| 18. RF provides many diverse SBRR materials!                          |
|                                                                       |
| 19. The Reading First program has helped me greatly to improve my     |
|     teaching abilities.                                               |
|                                                                       |
| 20. The Reading First program has impacted my teaching a great deal.  |
|     This program has enabled me to become a more effective educator.  |
|     Thanks Reading First.                                             |
|                                                                       |
| 21. The reading first program is wonderful. It has had a great impact |
|     on the way I teach reading.                                       |
|                                                                       |
| 22. The RF materials and resources are most useful. \[name\],         |
|     \[name\] and \[name\] are my favorites. I think you should        |
|     consider giving our LL a pay raise. She works extremely hard.     |
|                                                                       |
| 23. The school\'s Reading First program has assisted me greatly in my |
|     ability to provide lower students with explicit instruction.      |
|                                                                       |
| 24. Tier 3 students are returning to the classroom excited about      |
|     reading because of the extra practice they are receiving daily.   |
|                                                                       |
| 25. \[name\] would not be as successful if it were not for the        |
|     leadership shown by our Literacy Leader and Principal. Together   |
|     they are an extremely effective team.                             |
|                                                                       |
| 26. Our literacy leader has much respect and trust among the teachers |
|     of our grade level, I know. I feel as though we can go to her for |
|     any questions that we have. She offers herself to model or help   |
|     in our classrooms as much as possible.                            |
|                                                                       |
| 27. I enjoy it!                                                       |
|                                                                       |
| 28. I feel that the Reading First program is working wonderfully at   |
|     our school. I have learned a great deal of information in the     |
|     last three years. I hope that every school in the United States   |
|     becomes a Reading First school. It has made a difference at       |
|     \[name\]!                                                         |
|                                                                       |
| 29. I think RF is the best thing that has happened to the reading     |
|     program at \[name\].                                              |
|                                                                       |
| 30. Reading First program has made a positive impact in the students  |
|     reading progress, and it has helped me in many ways to be a       |
|     better teacher in the teaching of reading.                        |
|                                                                       |
| 31. Using \[name\] in kindergarten has been awesome for students of   |
|     all ability levels!                                               |
|                                                                       |
| 32. I have enjoyed the Reading First Program. I have learned new and  |
|     exciting activities to use in my classroom.                       |
|                                                                       |
| 33. I really enjoy our Reading First Program. The students also seem  |
|     to enjoy reading.                                                 |
|                                                                       |
| 34. Would like to continue to be a part of the Reading First          |
|     Initiative                                                        |
|                                                                       |
| 35. Great impact on our school.                                       |
|                                                                       |
| 36. It has been a pleasure working with the reading first leader and  |
|     learning many new techniques to help children with their reading  |
|     skills.                                                           |
+-----------------------------------------------------------------------+

**Table 8: Continued**

+-----------------------------------------------------------------------+
| ***[Positive Comments]{.underline}***                                 |
|                                                                       |
| **Please provide any additional comments you may have pertaining to   |
| your school's literacy program.**                                     |
+-----------------------------------------------------------------------+
| *[Responses]{.underline}*                                             |
+-----------------------------------------------------------------------+
| 37. The small groups give more individual attention to each child.    |
|     The invention groups also have the children in smaller groups for |
|     reading.                                                          |
|                                                                       |
| 38. The Reading First program is a great tool for students with       |
|     special needs.                                                    |
|                                                                       |
| 39. We have an outstanding principal and literacy specialist who are  |
|     doing everything to make sure teachers are comfortable doing what |
|     they are hired to do                                              |
|                                                                       |
| 40. Because of RF, I feel very confident in my ability to teach my    |
|     students. The information from our Cadre Sessions provided me     |
|     with valuable tools with which to work. I am a better teacher     |
|     because of RF.                                                    |
|                                                                       |
| 41. Our Cadre training conducted by \[name\] has been very helpful    |
|     and has impacted my teaching.                                     |
|                                                                       |
| 42. Reading First is a very good program                              |
|                                                                       |
| 43. The administration, Literacy Leader, and Cadre are extremely      |
|     supportive of the staff and are always available to assist the RF |
|     faculty in any way that promotes academic success.                |
+-----------------------------------------------------------------------+

+-----------------------------------------------------------------------+
| ***[Neutral or Negative Comments]{.underline}***                      |
|                                                                       |
| **Please provide any additional comments you may have pertaining to   |
| your school's literacy program.**                                     |
+-----------------------------------------------------------------------+
| *[Responses]{.underline}*                                             |
+-----------------------------------------------------------------------+
| 1.  Teachers do not have nearly the planning time to adequately       |
|     develop lessons. There is a deep sense of pressure. We are also a |
|     \[name\] a magnet program that also requires a lot of time.       |
|                                                                       |
| 2.  Our Administration does a great job providing us with as much     |
|     planning time as possible but the demands of other meetings,      |
|     data, special ed. and other classroom preparations means that     |
|     much time before and after school is required to meet Reading     |
|     First demands adequately.                                         |
|                                                                       |
| 3.  Having a hard time getting 90 PD hours this year.                 |
|                                                                       |
| 4.  The worst part of being involved in the Reading First program are |
|     the 90 professional development hours every year!                 |
|                                                                       |
| 5.  One person that needs to be added to the program to assist with   |
|     the workload of the literacy leader is a literacy leader          |
|     assistant.                                                        |
|                                                                       |
| 6.  Please consider starting Tier 3 instruction at the end of August  |
|     in order to better serve the students.                            |
|                                                                       |
| 7.  We need more time built into our day for planning to develop      |
|     plans for our teachers in the Reading First 90 minute block.      |
|                                                                       |
| 8.  Wonderful program. My special education students are showing      |
|     great improvements in reading. However, I do not have enough time |
|     to plan at school and seem to be taking more and more material    |
|     home with me.                                                     |
|                                                                       |
| 9.  I believe that this program is worthwhile but too much emphasis   |
|     is placed on Dibels scores and I have found them to be unreliable |
|     at times. Other forms of authentic assessment should be included. |
|                                                                       |
| 10. I would like more feedback from Lots. Too much paperwork. It      |
|     needs to be a part to deal with student\'s social skills. Working |
|     in small groups students need to learn how to work with each      |
|     other. (sic)                                                      |
|                                                                       |
| 11. Literacy leaders should have help in working with children in the |
|     low tier groups.                                                  |
|                                                                       |
| 12. Teachers and literacy leaders are expected to do the impossible.  |
|                                                                       |
| 13. Materials are readily available, however, there is so much that   |
|     no one has the opportunity or time to really get into the         |
|     materials to see how they can best be utilized in the class. We   |
|     also need better storage so items don\'t get packed away and not  |
|     used simply because it has been forgotten.                        |
|                                                                       |
| 14. Overloaded with paperwork, not enough planning time, and more TAs |
|     would be wonderful. The time for DIBELS is not allotted by RF and |
|     interferes with other subjects. As a teacher, I believe all       |
|     subjects should be addressed sufficiently. The program continues  |
|     to change and not necessarily for the better. Teachers are often  |
|     overwhelmed.                                                      |
|                                                                       |
| 15. Some teachers are frustrated with all the requirements of the     |
|     Reading First Program.                                            |
|                                                                       |
| 16. Teachers have kept up with their end of the bargain in            |
|     implementing the Reading First Program. Reading First has not     |
|     kept their end of the bargain. We are expected to do the job of   |
|     two people. That is very stressful and almost impossible.         |
|     According to the program, as a Reading First School, we should    |
|     have additional help with our intensive students. Teachers are    |
|     held accountable. When are the people who are implementing the    |
|     program held accountable.                                         |
|                                                                       |
| 17. Visitors come in the classroom and observe but never offer        |
|     individual feedback.                                              |
|                                                                       |
| 18. We need more funds to staff more interventionists to address our  |
|     needs.                                                            |
+-----------------------------------------------------------------------+

**Table 9: Control Teacher Responses**

+-----------------------------------------------------------------------+
| ***[Positive Comments]{.underline}***                                 |
|                                                                       |
| **Please provide any additional comments you may have pertaining to   |
| your school's literacy program.**                                     |
+-----------------------------------------------------------------------+
| *[Responses]{.underline}*                                             |
+-----------------------------------------------------------------------+
| 1.  I've seen wonderful success in children reading in kindergarten   |
|     due to the literacy program \[name\] embraces.                    |
|                                                                       |
| 2.  I have seen success with using guided reading techniques and      |
|     leveled books.                                                    |
|                                                                       |
| 3.  I feel we have an excellent literacy program.                     |
|                                                                       |
| 4.  I feel that overall we are doing a great job. Parental support    |
|     isn't where it should be. We are definitely understaffed, but     |
|     that is beyond our control. Student behavior could be a factor,   |
|     which could be adversely affected if each teacher had a full-time |
|     assistant to assist those in need. Those are usually the one's    |
|     creating the disruptions. But overall, our school is on top of    |
|     things.                                                           |
|                                                                       |
| 5.  We have a very supportive administration, here at \[name\], which |
|     is readily available and motivates their teaching staff.          |
|                                                                       |
| 6.  This year we started \[name\] and students are motivated with it! |
|                                                                       |
| 7.  \[name\]'s literacy program promotes our students to the utmost   |
|     proficient level. We take great pride in the students academic    |
|     abilities and of students with special needs an exceptional and   |
|     quality education. (sic)                                          |
|                                                                       |
| > We have a great literacy program. All of our faculty and staff are  |
| > dedicated and want every child to be successful.                    |
+-----------------------------------------------------------------------+
|                                                                       |
+-----------------------------------------------------------------------+
| ***[Neutral or Negative Comments]{.underline}***                      |
|                                                                       |
| **Please provide any additional comments you may have pertaining to   |
| your school's literacy program.**                                     |
+-----------------------------------------------------------------------+
| *[Responses]{.underline}*                                             |
+-----------------------------------------------------------------------+
| ##### I look forward to purchasing additional materials to enric      |
| h my centers. I would love to be a part of Reading First training, be |
| cause ideas for workstations help a great deal. Some of my colleagues |
|  came from a Reading First school, and have given us wonderful ideas. |
|                                                                       |
| 1.  We could use more guided reading books that flow with the unit.   |
|                                                                       |
| 2.  I do not like \[name\]. It does not have adequate decodable       |
|     readers (specifically take home decodable readers) for the lower  |
|     grades. It does not incorporate enough phonics. It is not         |
|     systemic enough in its approach. It does not spiral, review       |
|     enough. It does not hit decoding skills hard enough. The sight    |
|     words do not tie into the theme of the week. Please look at       |
|     \[name\]. It is an excellent reading series.                      |
|                                                                       |
| 3.  Students need more improvement in reading and writing skills.     |
|     Speech and Drama should be a part of the curriculum. AS a teacher |
|     this is a project that is being included to motivate and          |
|     encourage students to express themselves more in order to improve |
|     these skills.                                                     |
|                                                                       |
| 4.  There is a lack of phonemic awareness and phonetic skills.        |
|                                                                       |
| 5.  Are we testing too much? Are we placing too much emphasis on      |
|     tests? Q#26---We live in the real world- we can only hope and     |
|     pray all children can read at or above grade level by 3^rd^       |
|     grade. I know for a fact -- no matter how hard you try some       |
|     children may not get there.                                       |
|                                                                       |
| 6.  There is so much "paperwork" that without a teaching assistance   |
|     it is almost impossible to fully implement any program.           |
|                                                                       |
| 7.  Computers need to be replaced with new ones.                      |
|                                                                       |
| 8.  #27 Student misbehavior will always effect a classroom regardless |
|     of the program being used. Maybe the question should be, "Does    |
|     your program minimize student misbehavior?" #26 Anyone who agrees |
|     to this statement is probably not living in reality. (sic)        |
|                                                                       |
| 9.  To my knowledge we do not implement a specific literacy program.  |
|                                                                       |
| 10. We do not have a literacy program.                                |
|                                                                       |
| 11. We have not had to access to materials/personnel that would       |
|     really create a literacy program. I feel that the new reading     |
|     system that we will get next year will change our school. I think |
|     if I had the new reading program, many of my disagree answers     |
|     would have been agree. (sic)                                      |
|                                                                       |
| 12. I do not feel that I have been trained enough in this literacy    |
|     program. I had three classes in college that taught me the same   |
|     information . It is still hard for me to apply what I have        |
|     learned. We should have a workshop that teaches, or reinforces,   |
|     the terms and concepts we have been taught in \[name\].           |
|                                                                       |
| 13. At this time, this is an unfair assessment because the teachers   |
|     at \[name\] are just beginning the training in the \[name\]       |
|     program. The fourth, fifth, and special education teachers        |
|     haven't been exposed to it. Second and third are just now         |
|     completing the course. Also, I feel our program will be           |
|     straightened with the new reading series we are currently         |
|     adopting. Lastly, I don't feel that is fair to be a teacher on    |
|     class to throw a new program at them mid year and expect to see   |
|     results. I have based my answers on the previous page on our      |
|     current reading program because at this time we don't have a      |
|     literacy program in place at our school.                          |
|                                                                       |
| 14. We do not have a literacy program. We have a reading program that |
|     is used daily.                                                    |
|                                                                       |
| 15. We all need more planning time.                                   |
|                                                                       |
| 16. I find many students lacking in decoding and encoding skills and  |
|     strategies. I think this is one draw back of \[name\]. On the     |
|     other hand, I think \[name\] is great for ESL students.           |
|                                                                       |
| > Our class sizes are too big. We are not supposed to have more than  |
| > 16 students per teacher and I usually have 24-25 students. #14 I    |
| > was taught these reading principles in college. #20- I have no      |
| > prior knowledge of previous reading programs to base this off of.   |
| > Best thing about \[name\]---data driven instruction. Worst things-  |
| > switching classes creates chaos, at least two weeks of instruction  |
| > is lost every quarter because of switching classes; I feel out of   |
| > touch with needs of my homeroom because of it; program is too       |
| > detail-oriented. There is not enough time to do it all and          |
| > creativity in teaching and learning is not encouraged. (sic)        |
+-----------------------------------------------------------------------+

***Inferential Results***

An inferential analysis was conducted based on the available
questionnaires returned: 262 from the treatment schools and 189 from the
control schools. A total of 27 questions compose the RFTQ, and each is
measured on a 5-point Likert Scale: 1=Strongly Disagree, 2=Disagree,
3=Neutral, 4=Agree, and 5=Strongly Agree. As the data for the RFTQ are
ordinarily scaled, it was theorized that the assumption of normality
would not be satisfied. In fact, the Shapiro-Wilk test for each of the
27 questions was statistically significant at an alpha level of .0018
(.05 divided by 27 questions), revealing that the data were not
normality distributed. Therefore, a non-parametric analysis, Wilcoxon
Mann Whitney, was employed to draw the statistical inference.

The Wilcoxon uses a procedure that orders the data from lowest to
highest, and assigns each data point a number starting from 1 to the
sample size. In the case of ties, each is assigned the average value of
the ranks. The statistical inference is based on these assigned rank
integers. In Table 10, the Wilcoxon outcomes for the RFTQ and the mean
score of the assigned rank integer are given for each group along with Z
statistics and p-values.

The alpha level was adjusted to 0.0018, taking 27 questions into
account. All of the RFTQ questions were rated more positively (mostly
Agree or Strongly Agree) in the treatment group as opposed to the
control group (see Table 10). The mean scores in the treatment group
were all at least 4.00 with the exception of three questions, while the
control group had over half of questions (15 of 27) with a mean below
4.00. In addition, there was a statistically significant difference
between groups in favor of the treatment group on all but three
questions, an indication of highly positive responses from the treatment
teachers.

**Table 10: RFTQ Items Depicting Significant Differences between
Treatment and Control Groups**

  ------------------------ --------------- ---------- ---------- ------------- ---------- ---------- --------- ---------
                           **Treatment**                         **Control**                         ***Z***   ***p***

  **RFTQ Items**           ***M***         ***SD***   ***Mean    ***M***       ***SD***   ***Mean              
                                                      Rank                                Rank                 
                                                      Score***                            Score***             

  I have a thorough        4.63            0.64       244.26     4.37          0.70       198.02     -4.32\*   \<.0001
  understanding of this                                                                                        
  school\'s RF program.                                                                                        

  I have received adequate 4.67            0.64       256.55     4.21          0.82       180.79     -7.01\*   \<.0001
  initial and ongoing                                                                                          
  professional                                                                                                 
  development/ training                                                                                        
  for implementation of my                                                                                     
  school\'s RF program.                                                                                        

  Professional development 4.60            0.62       248.75     4.29          0.73       191.72     -5.22\*   \<.0001
  provided by the                                                                                              
  Tennessee Reading Cadre                                                                                      
  has been valuable.                                                                                           

  Professional development 4.34            0.71       237.64     4.14          0.73       203.55     -3.03     .0024
  provided by the                                                                                              
  Tennessee Department of                                                                                      
  Education has been                                                                                           
  valuable.                                                                                                    

  Guidance and support     4.62            0.66       258.10     3.97          1.01       171.73     -7.81\*   \<.0001
  provided by our                                                                                              
  school\'s Literacy                                                                                           
  Leader have helped our                                                                                       
  school implement its RF                                                                                      
  program.                                                                                                     

  The principal is an      4.50            0.69       239.61     4.29          0.75       204.52     -3.16\*   .0016
  effective instructional                                                                                      
  leader.                                                                                                      

  Teachers are given       3.59            1.23       240.16     3.22          1.23       201.33     -3.24\*   .0012
  sufficient planning time                                                                                     
  to implement our RF                                                                                          
  program.                                                                                                     

  All instructional        4.74            0.46       265.75     4.18          0.80       170.90     -8.78\*   \<.0001
  elements of RF at our                                                                                        
  school - assessments,                                                                                        
  programs, supplemental                                                                                       
  materials - are based on                                                                                     
  scientifically-based                                                                                         
  reading research.                                                                                            

  Because of RF,           4.76            0.49       261.38     4.25          0.78       176.96     -8.01\*   \<.0001
  systematic and explicit                                                                                      
  instruction in phonemic                                                                                      
  awareness, phonics,                                                                                          
  vocabulary, fluency, and                                                                                     
  comprehension occurs                                                                                         
  daily in our school's                                                                                        
  K-3 classrooms.                                                                                              

  Because of our RF,       4.66            0.53       252.82     4.29          0.70       187.43     -6.02\*   \<.0001
  teachers use assessments                                                                                     
  to inform instructional                                                                                      
  decisions and monitor                                                                                        
  student achievement.                                                                                         

  Scientifically-based     4.67            0.58       270.84     3.99          0.83       162.32     -9.73\*   \<.0001
  instructional materials                                                                                      
  (books, assessments, and                                                                                     
  other resources) needed                                                                                      
  to implement our RF                                                                                          
  program are readily                                                                                          
  available.                                                                                                   

  Our school has           4.06            1.09       237.28     3.81          1.09       203.88     -2.85     .0043
  sufficient faculty and                                                                                       
  staff to fully implement                                                                                     
  its RF program.                                                                                              
  ------------------------ --------------- ---------- ---------- ------------- ---------- ---------- --------- ---------

**Implementation Benchmark Ratings**

***Descriptive Results***

During the benchmark process, a team from each of the Reading First
schools developed goals with measurable criteria and then classified the
school's progress towards attaining those goals. Goals were established
for the categories of: Curriculum, Instruction, and Organization. Each
category had multiple components or elements in which the benchmarking
team measured progress. Each component was deemed to be in: Phase 1 (the
lowest level of implementation), Phase 2 (some level of attainment, yet
not fully realized), or Phase 3 (fully attained goal). Only treatment
(Reading First) schools participated in the benchmarking process. The
schools selected for the research study were either Round 1 or Round 2
schools. There were 10 schools which were Round I schools; meaning that
during the 2006-2007 school year, these schools were in their fourth
year of implementation of Reading First. The remaining two schools were
designated as Round 2 schools and were in the third year of
implementation. Therefore, the Reading First schools had generally
trained teachers, who were experienced in Reading First, and who were
fully implementing the program. Not surprisingly, most benchmark teams
considered themselves to have reached the Phase 3 level of attainment
for most of their goals.

For the 12 schools in the research study, 11 reported to be in Phase 3
for the *Curriculum* benchmark goal, indicating that
scientifically-based reading program components are cross-referenced to
state and local learning standards and monitored regularly for
effectiveness in increasing student achievement. Additionally, a Phase 3
level also indicates that any modifications in curriculum are made to
accommodate the requirements of Reading First. One school indicated
being in Phase 2 for the *Curriculum* benchmark goal, which would
suggest that the Reading First core, supplementary, and intervention
curriculum content is based on scientific research and is aligned with
state performance standards and with the most current assessment
results.

Results from the RFTQ supported the benchmark rankings. Almost all of
the respondents (98.9%) of the Reading First teachers agreed that all
instructional elements were based on scientifically-based reading
research; in addition; 96.2% agreed that elements of Reading First were
effectively integrated to help meet school improvement goals. Over 80
percent (83.2%) responded positively that because of Reading First,
teachers spent more time together developing curriculum and planning
instruction.

Regarding the overall *Instruction* benchmark goal, 9 of the 12
treatment schools reported they were in Phase 3, which indicates that
all teachers routinely and skillfully use research-based teaching
strategies for the entire reading instruction time. Additionally, this
would indicate that components of the reading program are taught in a
coherent and systematic fashion. The remaining three treatment schools
reported being in Phase 2 for the overall *Instruction* goal, which
suggests most teachers use a variety of research-based teaching
strategies to teach reading. This would also indicate that systems are
in place to monitor/evaluate students' skill levels and provide
appropriate coaching and feedback to students. A review of the specific
components of *Instruction* (phonemic awareness, phonics, fluency,
vocabulary, text comprehension, and assessment), showed wider variety of
perceived attainment among the schools. Fluency activities were ranked
as Phase 3 for all schools, while text comprehension was only rated as a
Phase 3 for three of the schools. The majority of teachers from the
treatment schools (97.7%) agreed that because of Reading First,
systematic and explicit instruction in the five essential elements of
reading (phonemic awareness, phonics, vocabulary, fluency, and
comprehension) occurs daily in their classrooms.

The review of LOT data provides a fuller context for interpreting
benchmarking goals. Although Reading First teachers utilized a wider
variety of instructional orientations and techniques compared with their
control counterparts, greater movement towards Phase 3 would be expected
for schools in their third and forth years of implementation. The
majority of Reading First teachers were either rarely or not observed
engaged in phonemic awareness or phonics instruction, yet just one-third
of the schools (4 schools) indicated a Phase 2 for this area; the
remaining schools indicated a Phase 3 level of attainment. This suggests
perhaps the benchmarking team overestimated instructional focus in this
area. A more accurate reflection of instruction was indicated in the
area of text comprehension, in which 9 of 12 schools reported a Phase 2
attainment. LOT data supported this finding with the majority of Reading
First teachers either never or rarely observed utilizing text
comprehension techniques.

For the *Organization* benchmark goal, 11 treatment schools reported
that they were in Phase 3 indicating that all major organizational
structures that support the school goals and the Reading First program
are in place and functioning appropriately and effectively. This would
be expected for schools that had received funding over multiple years.
The area of greatest concern appeared to be parental involvement. Six of
the schools reported a Phase 2 attainment in this area, and this is
further reflected in the teacher questionnaire. One of the least
positive survey items was: "*Because of our RF program, parents are move
involved in the literacy program of this school*"---there were 56.5% of
the teachers who agreed with this statement, 25.6% that were neutral,
and 17.9% who disagreed. Table 11 summarizes the benchmark results.

**Table 11: Reading First Treatment Schools Benchmarks**

  ------------ -------------- ----------- ------------ ---------------- ----------------- ------------------
  **School**   **District**   **Years in\ **Grades**   **Curriculum**   **Instruction**   **Organization**
                              Reading                                                     
                              First \***                                                  

  Collinwood   Wayne          4           K-4          Phase 3          Phase 3           Phase 3

  Inskip       Knox           3           K-5          Phase 3          Phase 3           Phase 3

  East Lincoln Tullahoma      4           K-5          Phase 3          Phase 3           Phase 3

  Parris South Hardin         4           K-5          Phase 2          Phase 2           Phase 3

  Spring Hill  Memphis City   4           K-5          Phase 3          Phase 3           Phase 3

  Westhaven    Memphis City   4           K-5          Phase 3          Phase 3           *Did not complete*

  Westwood     Memphis City   4           K-6          Phase 3          Phase 3           Phase 3

  Hamilton     Memphis City   3           K-5          Phase 3          Phase 3           Phase 3

  Orchard Knob Hamilton       4           K-5          Phase 3          Phase 2           Phase 3

  Battle       Hamilton       4           K-5          Phase 3          Phase 2           Phase 3

  Arlington    Madison        4           K-4          Phase 3          Phase 3           Phase 3

  North        Madison        4           K-4          Phase 3          Phase 3           Phase 3
  Parkway                                                                                 
  ------------ -------------- ----------- ------------ ---------------- ----------------- ------------------

\*For the 2006-2007 school year

**LITERACY TEST RESULTS**

**Participants**

A total of 1,716 Kindergarten students from 24 schools were involved in
the study. These students were divided nearly evenly into two groups:
the Reading First group had a sample size of 861; and the control group
had a sample size of 855. Schools were paired according to similar
characteristics such as size, location, and overall socioeconomic
status. Of the 1,716 subjects, a little more than half (53%) were male
(with female students comprising 47% of the sample). The majority of
students were Black (65%). The second largest ethnic group was White
students at 32% of the sample. The remaining 3% were composed of
Hispanic (2%), Indian (0.24%) and Asian (0.06%) students. The
demographics for both groups are shown in Table 12, and as can be seen,
were very similar.

**Table 12: Demographics by Group**

  -------------- -------------- ------------------ --------------- ------------------
                 **Reading                         **Non-Reading   
                 First Group**                     First Group**   

                 ***N***        ***Percentage***   ***N***         ***Percentage***

  Total Sample   861            50.17              855             49.83

  Male           476            55.35              438             51.23

  Female         384            44.65              417             48.77

  Black          542            63.69              534             66.67

  White          274            32.20              256             31.96

  Hispanic       32             3.76               9               1.12

  Indian         2              0.24               2               0.25

  Asian          1              0.12               0               0.00
  -------------- -------------- ------------------ --------------- ------------------

**Peabody Picture Vocabulary Test (PPVT) Analysis**

**Methodology**

Kindergarten students were assessed with the PPVT during the fall
(pretest) and spring (posttest) of the 2006-07 school year. To gauge
academic achievement over the course of the school year and to assess
the Reading First program's effect on achievement, a rigorous
Hierarchical Linear Model (HLM) was employed due to the tool's
usefulness in handling dependent observations and serial correlations.
Specifically, HLM is a regression model catering to data that are nested
in nature, such as students nested within classrooms, and classrooms
nested within schools, as in this research study.

The key advantage of HLM is that it allows for investigating individual
student-level outcomes by including multiple observations of achievement
and relating these observations to individual and school
characteristics. During the statistical analysis process, the students'
post-PPVT scores were related to student and school characteristics by a
series of regression functions (Level-1 and Level-2 models), which
yielded the impact of the pre-PPVT assessment as well as other key
determinants (e.g., race or gender). As a result, the Level-1 model was
employed to investigate the influence of individual characteristics and
pre-PPVT assessment scores on expected post-PPVT achievement. The
Level-2 model then addressed the impact of school characteristics (i.e.
Reading First or control group) on post-PPVT outcomes.

A total of 1,427 Kindergarten students with pre and posttest PPVT scores
were included in the study. A two-level HLM model was estimated to
examine the relationship between student characteristics and
participation in Reading First on PPVT achievement. As previously
described, the Level-1 model related students' demographic
characteristics and pre-assessment PPVT scores to post-PPVT achievement.
Three dichotomous variables were included in the Level-1 model. Gender
was dummy coded where female had a value of one and male zero. Three
ethnic groups: White, Black and Other were taken into consideration to
investigate their impact on achievement, with White set as a reference
group. As a result, two dummy variables, Black and Other, were coded
representing three ethnic groups: White (black=0 and other=0), Black
(black=1 and other=0) and other ethic groups (black=0 and other=1).

At Level-2, the intercept and the slope of the Level-1 model was a
function of whether or not students participated in the Reading First
program. It only has one predictor: A dummy variable ("group") took on a
value of zero to represent the control group and a value of one for the
Reading First group. The estimated Intraclass Correlation Coefficient
(ICC) was .14, which indicated there was considerable variability among
students within schools, further justifying that HLM was an appropriate
analytical tool in this study.

The final model shown in the equations below is a reduced model obtained
from the full model where both the Level-1 PPVT slope and BLACK slope
were random varying functions. The estimated regression coefficients are
presented in Table 13. The Level-1 intercept was set as non-random
varying due to its insignificant variable component. In addition, the
residual parameter variance for SEX was set to zero. As a result, both
the pre-PPVT slope and the BLACK slope were allowed to vary randomly.
The reliability estimates of the random Level-1 coefficients for the
pre-PPVT and BLACK slopes were 0.72 and 0.65 respectively, strong
evidence of discriminating between students in terms of pre-PPVT
achievement and ethnic background (Black or non-Black). The Level-2
variance component revealed that there was significant variability among
students in pre-PPVT achievement (*χ^2^*= 76.63, *df*=19, *p*\<.001) and
between students who are Black and those who are not (*χ^2^*= 69.71,
*df*=20, *p*\<.001).

![](media/image3.wmf){width="5.798611111111111in"
height="2.0527777777777776in"}

Where

-   *β~0j,~* *β~1j,~ β~2j,\ ~*and *β~3j~* are the Level-1 regression
    coefficients: Intercept and slopes respectively. *β~0j\ ~*represents
    the expected post-PPVT achievement adjusted for pre-PPVT
    achievement. *β~1j,~ β~2j,\ ~*and *β~3j\ ~*relate the
    characteristics on pre-PPVT, gender, and ethnic information to the
    post-PPVT achievement by defining the rate of change of the
    post-PPVT with respect to these traits.

-   γ*~00,~* γ*~10,~* γ*~11,~*γ*~20,\ ­~and* γ*~30~ are* Level-2
    regression coefficients;

-   γ*~ij,~* μ~*1j*~ and μ*~3j~* are random errors associated with the
    Level-1 model slopes;

-   PPVT06 is the standard score of the PPVT assessment in fall 2006;

-   PPVT07 is the standard score of the PPVT assessment in spring 2007;

-   GENDER indicates female if it equals 1 and male if it equals 0;

-   BLACK is an indicator of the Black ethnic group: The student is
    Black when BLACK=1, and non-Black when BLACK=0;

-   OTHER is an indicator of other ethnic groups, or ethnic groups other
    than White or Black, when OTHER=1, otherwise OTHER=0;

-   GROUP predictor is an indicator of the treatment condition,
    0=Control group and 1=Reading First.

Please note that OTHER was not a significant determinant factor in the
PPVT achievement estimation, and was thus dropped from the model. As
shown in Table 13, the expected post-PPVT achievement was 91.62
(γ~00~=91.62, *t* =188.14, *p* \<.001), an adjusted post-mean
achievement given the grand mean of pre-PPVT assessment regardless of
the students' demographic background. Female students had a stronger
slope as opposed to male students in the estimation of the expected
post-PPVT (γ~20~=0.93, *t* =2.64, *p =*.009). The coefficient of the
Black slope (γ~30~=-2.39, *t* =-4.07, *p* =.001) came into the model
negatively, indicating that being a black student, the expected
post-PPVT score would have about a 2.39 weaker slope relative to that of
the non-Black students, when everything else is held constant.

The intercept of the PPVT slope was 0.67 (γ~10~=0.67, *t* =24.19, *p*
\<.001), evidence of higher than expected post-PPVT achievement given a
greater positive difference between the pre-PPVT assessment and the
grand mean of pre-PPVT, irrespective of school. However, this
variability was significantly dependent upon the treatment condition
(γ~11~=-0.07, *t* =-2.17, *p =*.041). Specifically, Reading First
students who had relatively lower pre-PPVT achievement than the grand
mean of pre-PPVT would have a stronger slope estimate than would the
control group counterparts. On the other hand, the Reading First
students who had a positive difference between the pre-PPVT and the
grand mean of pre-PPVT tended to have a weaker slope as opposed to the
control counterparts. The pattern is depicted in Figures 1 through 3.
Notably, for Figure 1, in the left half of the graph (representing low
achievement in the pretest PPVT) the line for Reading First students is
above the one for the control students while on the right side of graph,
the line for Reading First students is below that of control students.
This is pronounced evidence of significantly positive impact of the
Reading First program on low performers.

The parallel lines in Figure 2 indicate that female students had higher
than expected pre-PPVT adjusted post-PPVT outcomes than male students.
As demonstrated in Figure 3, non-Black students had a higher slope
estimate compared to that of Black students. Similarly, an examination
of the Q-Q plot of the Level-1 residuals (Figure 4) showed an
approximately normal distribution, upholding the tenability of the
assumption of a Gaussian distribution of Level-1 errors. A violation of
this assumption could adversely affect the standard error estimations.

**Summary of PPVT Analysis Outcomes**

The analysis yielded several important findings. First, all students,
regardless of group membership, posted substantial gains in PPVT scores
over the 2006-07 school year. Furthermore, Reading First students who
were lower performers at the pretest assessment tended to have higher
post-achievement relative to control counterparts, while in general, the
high performers on the pretest assessment tended to have high post-PPVT
achievement irrespective of group membership (i.e., Reading First or
Control). Additional conclusions to be reached include the following:

-   Variables such as ethnic group and gender were relevant for
    predicting PPVT achievement.

-   In general, non-Black students outperformed Black students when all
    other variables were held constant.

-   Overall, female students outpaced male students holding other
    factors equal.

```{=html}
<!-- -->
```
-   Relative to control counterparts, students in the RF program
    performed lower at the pretest compared to the overall average, but
    tended to achieve higher scores on the post-PPVT assessment.

**Table 13: Reading First Effect on Growth in Student PPVT**

  -------------------------- ----------------- ------------- --------- ---------- ---------
  **Fixed effect**           **Coefficient**   **S.E.^1^**   ***t***   ***df***   ***P***

  Intercept                                                                       

  Intercept (γ~00~)          91.62             0.49          188.14    1422       0.000

  PPVT Slope                                                                      

  Intercept (γ~10~)          0.67              0.03          24.19     22         0.000

  ^4^Group (γ~11~)           -0.07             0.03          -2.17     22         0.041

  ^2^SEX Slope                                                                    

  Intercept (γ~20~)          0.93              0.35          2.64      1422       0.009

  ^3^BLACK Slope                                                                  

  Intercept (γ~30~)          -2.39             0.59          -4.07     23         0.001
  -------------------------- ----------------- ------------- --------- ---------- ---------

~1:\ Robust\ Standard\ Error~

~2:\ 0=Male,\ 1=Female~

~3:\ 0=White\ or\ other\ ethnic\ groups,\ 1=Black~

~4:\ 0=\ Control\ group,\ 1=\ Reading\ First~

![](media/image4.wmf){width="6.497222222222222in"
height="4.379166666666666in"}

**Figure 1: PPVT Achievement by Group Controlling Black and Gender**

![](media/image5.wmf){width="6.497222222222222in"
height="4.379166666666666in"}

**Figure 2: PPVT Achievement by Group and Gender Controlling Black**

![](media/image6.wmf){width="6.497222222222222in"
height="4.379166666666666in"}

**Figure 3: PPVT Achievement by Group and Black Controlling Gender**

![](media/image7.wmf){width="6.5in" height="4.874305555555556in"}

**Figure 4: Q-Q Plot of Level-1Residuals**

**DIBELS TEST RESULTS**

**Methodology**

DIBELS is composed of three subtests: Letter Naming Fluency (LNF),
Phoneme Segmentation Fluency (PSF), and Nonsense Word Fluency (NWF).
Kindergarten students were tested in the spring of 2007. Preliminary
analysis indicated that a considerable amount of variability existed
among students within schools as manifested by an approximate ICC of
.18, a value greater than 0.10, the threshold for HLM analysis. As a
result, an HLM, controlling for variations among students and schools,
was the analysis tool employed. A series of HLM models were estimated
for the three DIBELS subtests. The procedures taken were similar to
those in the PPVT achievement analysis. The details of the analysis and
the outcomes are presented by subtest.

***[Letter Naming Fluency]{.underline}***

![](media/image8.wmf){width="5.798611111111111in"
height="1.7826388888888889in"}

Where

-   β*~0j~* and β*~1j~* are the Level-1 regression coefficients:
    Intercept and slope respectively. β*~0j~* is the expected LNF
    achievement given the centered PPVT grand mean, that is, the
    expected value of student *i* at school *j* with a PPVT score equal
    to the grand mean of all the PPVT assessments;

-   γ*~00,~* γ*~10,~* γ*~11,~* and γ*~20,~* are the Level-2 regression
    coefficients. γ*~00~* is the adjusted mean LNF achievement over
    average PPVT achievement across schools. γ*~10~* is the average PPVT
    regression slope across those schools. γ*~11~* is the average
    regression slope for the Reading First schools given the PPVT
    pre-assessment. γ*~20~* is the average sex regression slope across
    those schools;

-   γ~*ij*~ and μ*~oj~* are random errors associated with the Level-1
    model and the intercept of Level-1 model respectively.

-   DLNF is a short name for DIBELS Letter Naming Fluency;

```{=html}
<!-- -->
```
-   PPVTSTD0*~ij~* is the pre-standard-PPVT score of the *i*th student
    at school *j*;

-   PPVTSTD0. is the grand-mean of the pre-standard-PPVT score of the
    *i*th student at school *j*;

-   SEX is a dummy coded predictor: 0=male and 1=female for student *i*
    at school *j*;

-   GROUP: 0= the Control group and 1=the Reading First group.

In the model, only the intercept is allowed to vary randomly over the
schools, the Level-2 units. For ease of interpretation, the PPVT
pre-assessment entered into model centering around the grand-mean (i.e.
deviations from a grand mean of PPVT irrespective of schools). Note that
ethnic groups were not significant determinant factors in the DIBELS
achievement estimation, thus only gender was included in the model.

As shown in Table 14, the grand mean (i.e., the average DIBELS LNF
achievement score) was 39.42 (γ~00~=39.42, *t* =30.69, *p* \<.001),
adjusted for the achievement in pre-PPVT across the schools regardless
of students' gender. The intercept of the PPVT slope (γ~10~=0.32, t
=10.36, *p\<.001)* represented how different the deviation of a student
was from the grand mean of PPVT on LNF achievement. The results
indicated that the higher the positive deviation was from the grand
mean, the stronger the slope of expected LNF achievement. Group value
(γ~11~=-0.15, t =-4.04, *p\<.001*) was negative and significantly
different from zero, discernable positive evidence of the Reading First
program on low performers. Basically, because Reading First took on the
value of 1 and the control 0, the negative coefficient was an indication
that the Reading First students who were low achievers at the
pre-assessment tended to have significantly stronger PPVT slopes, on
average, than did the control group low achievers. However, this is not
the case for high performers, who achieved higher than average on the
pre-PPVT assessment. Specifically, these Reading First students tended
to have a significantly weaker slope than the control students. Note
that students who had a pre-PPVT score less than the grand mean (which
is the overall mean irrespective of group membership) fell within the
category of low achievers.

The slope of gender (γ~20~=3.13, *t* =3.73, *p \<*.001) revealed that
the female students had an average slope 3.10 degrees higher than that
of males. These trends are reinforced by the graphic display of DIBELS
achievement in Figures 5 and 6, where the achievement line of Reading
First lies above the line of the control group when the students
performed low at pre-PPVT. This pattern depicting the relationship
between DIBELS LNF achievement, group membership, and gender remained
consistent for the Phoneme Segmentation Fluency and Nonsense Word
Fluency subtests, as displayed in Figures 7 through 10.

**Table 14: R*eading First Effect on DIBELS ***

  ------------------------- ----------------- --------------- --------- ---------- ---------
  **Fixed effect**          **Coefficient**   ***S.E.^1^***   ***t***   ***df***   ***p***

  Letter Naming Fluency                                                            

  Intercept                                                                        

  Intercept (γ~00~)         39.42             1.28            30.69     23         0.000

  PPVTSTD Slope                                                                    

  Intercept (γ~10~)         0.32              0.03            10.36     1423       0.000

  ^2^Group (γ~11~)          -0.15             0.04            -4.04     1423       0.000

  Sex Slope                                                                        

  Intercept (γ~20~)         3.13              0.84            3.73      1423       0.000

  Phoneme Segmentation                                                             
  Fluency                                                                          

  Intercept                                                                        

  Intercept (γ~00~)         37.52             2.03            18.50     23         0.000

  PPVTSTD Slope                                                                    

  Intercept (γ~10~)         0.27              0.03            8.50      1423       0.000

  ^2^Group (γ~11~)          -0.14             0.04            -3.65     1423       0.000

  Sex Slope                                                                        

  Intercept (γ~20~)         5.22              0.95            5.50      1423       0.000

  Nonsense Word Fluency                                                            

  Intercept                                                                        

  Intercept (γ~00~)         27.94             1.23            22.72     23         0.000

  PPVTSTD Slope                                                                    

  Intercept (γ~10~)         0.32              0.04            7.41      1423       0.000

  ^2^Group (γ~11~)          -0.14             0.05            -2.61     1423       0.010

  Sex Slope                                                                        

  Intercept (γ~20~)         3.34              0.87            3.83      1423       0.000
  ------------------------- ----------------- --------------- --------- ---------- ---------

~1:\ Robust\ Standard\ Error~

~2:\ 0=Control\ group,\ 1=Reading\ First\ group~

![](media/image9.wmf){width="6.497222222222222in"
height="4.379166666666666in"}

**Figure 5: Letter Naming Fluency by Group**

![](media/image10.wmf){width="6.497222222222222in"
height="4.379166666666666in"}

**Figure 6: Letter Naming Fluency by Group by Gender**

*[Phoneme Segmentation Fluency]{.underline}*

A model similar to that utilized for LNF was employed for Phoneme
Segmentation Fluency and is shown below:

![](media/image11.wmf){width="5.798611111111111in"
height="1.851388888888889in"}

Where

-   β*~0j~* and β*~1j~* are the Level-1 regression coefficients:
    Intercept and slope respectively. β*~0j~* is the expected PSF
    achievement given the centered PPVT grand mean, that is, the
    expected value of a student *i* at school *j* with a PPVT score
    equal to the grand mean of all the PPVT assessments;

-   γ*~00,~* γ*~10,~* γ*~11,~* and γ*~20,~* are the Level-2 regression
    coefficients. γ*~00~* is the adjusted mean PSF achievement over
    average PPVT achievement across schools. γ*~10~* is the average PPVT
    regression slope across those schools. γ*~11~* is the average
    regression slope for the Reading First schools given the PPVT
    pre-assessment. γ*~20~* is the average sex regression slope across
    those schools;

-   γ~*ij*~ and μ*~oj~* are random errors associated with the Level-1
    model and the intercept of the Level-1 model respectively;

-   DPSF is a short name for DIBELS Phoneme Segment Fluency;

```{=html}
<!-- -->
```
-   PPVTSTD0*~ij~* is the pre-standard-PPVT score of the *i*th student
    at school *j*;

-   PPVTSTD0. is the grand-mean of the pre-standard-PPVT score of the
    *i*th student at school *j*;

-   SEX is a dummy coded predictor: 0=male and 1=female for student *i*
    at school *j*;

-   GROUP: 0= the Control group and 1=the Reading First group.

As seen in Table 14, the regression outcomes indicated that the mean
Phoneme Segmentation Fluency score was 37.52 (γ~00~=37.52 *t* =18.50,
*p* \<.001) given an overall average of PPVT pre-assessment. The female
students had an average achievement slope 5.18 degrees higher than males
(γ~01~=5.22, *t* =5.50, *p* \<.001). The intercept of the PPVT slope
(γ~10~=0.27, t =8.50, *p\<.001)* indicated that the higher the positive
deviation from the PPVT grand mean, the higher the slope of PSF
achievement. Group value (γ~11~=-0.14, t =-3.65, *p\<.001*) was
significantly different from zero and led to two conclusions. First,
Reading First low performers at the pre-assessment tended to have a
stronger PPVT slope, on average, than did the control group low
performers. Second, the Reading First high performers at the
pre-assessment tended to have a weaker expected PPVT slope, on average,
than did the control group high performers.

![](media/image12.wmf){width="6.497222222222222in"
height="4.379166666666666in"}

**Figure 7: Phoneme Segmentation Fluency by Group**

![](media/image13.wmf){width="6.497222222222222in"
height="4.379166666666666in"}

**Figure 8: Phoneme Segmentation Fluency by Group by Gender**

*[Nonsense Word Fluency]{.underline}*

Similarly to the other models, the HLM model for Nonsense Word Fluency
is outlined below. As with the other models, only the intercept was
allowed to vary randomly in the model.

![](media/image14.wmf){width="5.798611111111111in"
height="1.707638888888889in"}

Where

-   β*~0j~* and β*~1j~* are the Level-1 regression coefficients:
    Intercept and slope respectively. β*~0j~* is the expected NWF
    achievement given the centered PPVT grand mean, that is, the
    expected value of a student *i* at school *j* with a PPVT score
    equal to the grand mean of all the PPVT assessments;

-   γ*~00,~* γ*~10,~* γ*~11,~* and γ*~20,~* are the Level-2 regression
    coefficients. γ*~00~* is the adjusted mean NWF achievement over
    average PPVT achievement across schools. γ*~10~* is the average PPVT
    regression slope across those schools. γ*~11~* is the average
    regression slope for the Reading First schools given PPVT
    pre-assessment. γ*~20~* is the average sex regression slope across
    those schools;

-   γ~*ij*~ and μ*~oj~* are random errors associated with the Level-1
    model and the intercept of the Level-1 model respectively;

-   DNWF is a short name for DIBELS Nonsense Word Fluency;

```{=html}
<!-- -->
```
-   PPVTSTD0*~ij~* is the pre-standard-PPVT score of the *i*th student
    at school *j*;

-   PPVTSTD0. is the grand-mean of the pre-standard-PPVT score of the
    *i*th student at school *j*;

-   SEX is a dummy coded predictor: 0=male and 1=female for student *i*
    at school *j*;

-   GROUP: 0= the Control group and 1=the Reading First group.

The outcomes (shown previously in Table 14) reveal that the expected
Nonsense Word Fluency score was 27.94 (γ~00~=27.94, *t* =22.72, *p
\<*.001) when the pre-assessment PPVT equals the overall average. Female
students increased at a significantly higher rate, an average of 3.34
(γ~20~=3.34, *t* =3.83, *p\<*.001) in the slope estimation of NWF
achievement. The intercept of the PPVT slope (γ~10~=0.32, t =7.41,
*p\<.001)* suggested that the greater the positive difference between
the PPVT assessment and the grand mean of PPVT, the higher the slope of
expected NWF achievement. Group value (γ~11~=-0.14, t =-2.61, *p=.010*)
was significantly different from zero indicating that Reading First low
performers tended to have enhanced PPVT slopes, on average, than did the
control group low performers while high performers tended to have lower
PPVT slopes.

![](media/image15.wmf){width="6.497222222222222in"
height="4.379166666666666in"}

**Figure 9: Nonsense Word Fluency by Group**

![](media/image16.wmf){width="6.497222222222222in"
height="4.379166666666666in"}

**Figure 10: Nonsense Word Fluency by Group by Gender**

*[Summary]{.underline}*

The tenability of the normality assumption was examined by the residuals
for each model and was found to be satisfactory. The reliability
estimates in the HLM model were strong: 0.85 for Letter Naming Fluency,
0.92 for Phoneme Segmentation Fluency, and 0.84 for Nonsense Word
Fluency, indicating strong evidence of discriminating between students
in terms of baseline achievement.

This analysis yielded several important findings. On average, a
discernable impact of being a RF program student was detected on DIBELS
achievement, taking into account the PPVT pre-assessment. That is, the
expected DIBELS achievement was relatively and significantly higher for
Reading First students who performed below the average compared to the
control students. Overall, the expected DIBELS achievement would be
anticipated to be high given a high positive difference between the
pre-PPVT assessment and the mean of PPVT. Further, this advantage is
particularly evident for female students with respect to males. In
addition, ethnic group membership did not play a role in predicting
DIBELS outcomes. It should be noted that there was only one testing
period (spring 2007) that was examined and included in the DIBELS study,
which renders limited power (i.e., ability to find a statistical
difference) in this longitudinal analysis.

**Conclusions**

The conclusions are presented in the context of the research questions.

**Do kindergarten through 3^rd^ grade students at Reading First schools
have significantly higher reading achievement scores than students at
control schools once socioeconomic status and pre-test scores are
controlled?**

The initial achievement results for kindergarten students in Reading
First schools were positive. All kindergarten students in both Reading
First and control schools were assessed with the PPVT in the fall of
2006, with students in the control group having slightly higher baseline
scores than those in the Reading First schools. Although all students,
regardless of group status, posted substantial gains in PPVT scores of
the 2006-2007 school year, those in the Reading First classrooms who
were lower performers during the fall assessment had higher spring PPVT
scores than those in the control group who were low performers. In
addition, Reading First students who scored below average on the fall
PPVT assessment scored statistically significantly higher on each of the
three DIBELS subtests than the control students who scored below average
on the fall PPVT assessment.

These results are promising and indicate that the instructional
techniques applied in the kindergarten classrooms are helping
low-performing students acquire the necessary pre-reading and beginning
reading skills to be successful in first grade. It should be noted that
while ethnicity did not appear to be a factor in the DIBELS subtests,
for the PPVT assessments, non-African American students tended to have
higher overall scores than the African American students. Also, female
students tended to outscore male students on the PPVT and DIBELS
subtests.

**Are Reading First program effects on reading achievement sustained
over time?**

Although initial results are positive concerning reading achievement,
years 2 and years 3 of the research study will be useful in determining
the sustainability of the impact of Reading First instruction. As the
2006-2007 kindergarten cohort are assessed as 1^st^ graders in 2008 and
2^nd^ graders in 2009, the achievement results will provide useful
information concerning the initial positive trends. The 2010 TCAP data
will also be useful in further understanding the lasting influence of
the Reading First program.

**Which Reading First program characteristics are associated with the
greatest increases in reading achievement?**

In year one of the study, in general, there were no substantial
differences between Reading First schools in terms of implementation,
literacy instruction, and teacher perceptions. Likewise, student
achievement was consistent, and positive, among the 12 Reading First
schools. Therefore for the baseline year, no strong conclusions can be
made regarding specific program characteristics as determinants for
reading achievement. The overall positive results on the DIBELS
subtests, especially for those children with initially low PPVT scores,
suggests that instruction that emphasized basic letter recognition and
phonemes had an impact on student growth and progress.

**Are classroom practices significantly different in Reading First
schools compared to control schools?**

The data collected from the LOT indicates that instructional practices
differed between Reading First classrooms and control classrooms. In
general teachers in Reading First classrooms utilized a wider variety of
instructional orientations than control teachers, including a greater
use of both learning centers and small group instruction. Reading First
teachers were also more likely to utilize a more varied array of
techniques in fluency instruction, vocabulary, and text comprehension.
Although alphabetic instruction was not emphasized in either Reading
First or control classrooms, Reading First teachers were more likely
than their control counterparts to apply phonemic awareness and phonics
instructional practices. The LOT data also suggested that all classrooms
had very positive learning environments with students actively engaged
and effective classroom management practices in use. In general, print
rich environments were also evident in all classrooms; however, the
Reading First classrooms were more likely to have evidence of student
writing on display. Additionally, Reading First teachers were observed
using a wider variety of materials in their classrooms than the control
teachers.

Results from the teacher questionnaires also suggested that classroom
practices differed between the treatment and control groups. For the
Reading First teachers, 95.0% agreed that Reading First had an impact on
instruction; while for control teachers, 61.4% agreed that their
literacy program had an impact on instruction. Reading First teachers
were also more positive concerning the availability of scientifically-
based materials (97.7% positively responding) than their control
counterparts (77.2% positively responding). Also, there were 98.9% of
the Reading First teachers who agreed that all elements of the literacy
program were based on scientifically-based reading research; while for
control teachers, 87.8% agreed this occurred in their schools.

**Do schools with fully implemented programs have greater gains in
reading achievement than those with lower levels of implementation?**

All of the schools in the research study have multiple years of
experience in implementing the Reading First program. Implementation
benchmarks suggested that the majority of both Round 1 and Round 2
schools considered themselves to be in Phase 3 implementation, or highly
implementing all aspects of the Reading First program. Therefore there
would appear to be few discernable differences in implementation of
Reading First among the 12 schools chosen for the study. With the
initial positive achievement scores across all Reading First research
schools, full implementation appears to have an impact on student
progress. However, without the ability to compare to lower-implementing
schools, stronger conclusions cannot be drawn.

**Summary**

The initial findings from year one of the research study suggest Reading
First is having a positive impact on both students and teachers. Reading
First appears to be fully implemented and well received by teachers and
staff at the research schools. Significantly, instruction provided at
Reading First schools appears to have an impact on early reading skills,
especially for those students who begin kindergarten at a disadvantage.
Years 2 and 3 of the research study will be necessary to understand the
more long-term impacts on student achievement and the consistency of the
early positive trends. As the study progresses it will also be important
to note:

-   Individual student success in more advanced reading skills

-   Ethnic and gender considerations in student achievement

-   Degree to which literacy observations reflect an increased use and
    variety of instructional techniques in fully implemented Reading
    First schools

-   Teacher perceptions of the program throughout its maturation

-   Discernable increases in parental involvement

**References**

Dunn, L. M., & Dunn, L. M. (2006). Peabody Picture Vocabulary Test
Fourth Edition. Circle Pines, MN: American Guidance Services, Inc.

Good, R. H., & Kaminski, R. A. (2003) *DIBELS Indicator of Basic Early
Literacy Skills.* Longmont: CO. Sopris West Educational Services.

National Reading Panel (2000). *Teaching children to read: an
evidence-based assessment of the scientific research literature on
reading and its implications for reading instruction.* Washington, D.C.:
National Institute of Child Health and Human Development.

National Research Council (1998). *Preventing reading difficulties in
young children.*

Committee on the Prevention of Reading Difficulties in Young Children.
C.E. Snow, M.S. Burns & P. Griffin (Eds.) Washington, D.C.: National
Academy Press.

No Child Left Behind Act (2001). Pub. L. No. 107-110, 107^th^ Congress.

Slavin, R. E., & Madden, N. A. (1993). *Multi-site replicated
experiments: An application to Success For All*. Paper presented at the
annual meeting of the American Educational Research Association,
Atlanta, GA.

Smith, L. J., Ross, S. M., & Grehan, A. W. (2002). Literacy Observation
Tool (LOT^©^). Memphis, TN: Center for Research in Educational Policy,
The University of Memphis.