STATEMENT OF WORK

Attachment A

Reading Excellence Act

School and Classroom Implementation and Impact Study (SCII)

# BACKGROUND

The Reading Excellence Act (REA) was authorized in 1999 by an amendment
to Title II of the Elementary and Secondary Education Act of 1965.

The central purpose of the Reading Excellence Act (REA) is to improve
children's reading achievement in high-poverty or low-performing
schools. The REA was also authorized to carry out the following
purposes:

-   Teach every child to read by the end of third grade.

-   Provide children in early childhood with the readiness skills and
    support they need to learn to read once they enter school.

-   Expand the number of high-quality family literacy programs.

-   Provide early intervention to children who are at risk of being
    identified for special education inappropriately.

-   Base instruction, including tutoring, on scientifically-based
    reading research.

    The law was passed for two major reasons. First, in recent years,
    findings from scientifically based reading research have provided
    compelling guidance for improved reading instruction. Second,
    national assessments have continued to show great need for improving
    reading instruction in many schools, especially high poverty
    schools.

    The REA program is aimed at changing teacher practice in grade K-3
    classrooms in high poverty and low-performing schools. The REA
    program makes competitive grants to states, which in turn hold
    competitions among specific types of districts for subgrants.
    Eligible districts include the following:

-   local school districts that have at least one school in Title I
    school improvement status;

-   districts with the highest or second highest percentages of poverty
    in the state; and

-   districts with the highest or second highest number of poor children
    in the state.

    Both state grantees and district subgrantees must carry out a
    variety of specific activities. Districts and schools must provide:

-   Professional development activities designed to improve the reading
    instruction practice of teachers and other instructional staff.

-   Early literacy intervention to children experiencing reading
    difficulties, including tutoring and other extended learning
    opportunities and kindergarten transition programs.

-   Family literacy services (e.g., parent and child interactive
    activities, early childhood education, adult literacy training, and
    parent education).

    Reading instruction must cover the six dimensions of reading in the
    REA's definition of reading. The REA's definition of reading is as
    follows:

    "The term 'reading' means a complex system of deriving meaning from
    print that requires all of the following:

A.  The skills and knowledge to understand how phonemes, or speech
    sounds, are connected to print.

B.  The ability to decode unfamiliar words.

C.  The ability to read fluently.

D.  Sufficient background information and vocabulary to foster reading
    comprehension.

E.  The development of appropriate active strategies to construct
    meaning from print.

F.  The development and maintenance of a motivation to read.

The REA also requires that reading instruction be based on
scientifically based reading research.

Congress appropriated \$260.0 million for the Reading Excellence Act in
FY 1999. The Department awarded \$238.8 million in grants to 17 states
in August 1999. States vary in their dates for awarding subgrants---they
range from January to April 2000. The REA office estimates that 1600
schools will be served by the program this year. There is no minimum or
maximum grant size, but grants must be sufficient to operate an REA
program for a two-year period. A new round of awards is expected to be
made in July, 2000.

The REA is authorized by an amendment to Title II of the ESEA (Sec.
101). The legislation requires a national evaluation, but does not
specify the types of information it must collect.

# OBJECTIVES

The central objectives of the study are to: make judgements about the
quality of school and classroom-level implementation of REA; describe
how REA programs are structured and how they operate; determine whether
schools that participate in REA are improving the quality of reading
instruction; and assess whether student outcomes are improving in REA
schools.

Within these central objectives, the study shall:

-   Describe the characteristics of a representative sample of REA
    schools and the districts in which they are located

-   Describe how REA programs are structured and what services REA
    funding supports

-   Determine what percentage of REA programs are well implemented and
    examine implementation, concentrating on professional development,
    teacher knowledge, and teacher practice

-   Provide REA states, subgrantees, and ED program managers with
    information on program implementation

-   Determine whether instruction conforms with research-based
    principles and is improving in REA schools

-   Determine whether students in REA schools are improving their
    reading achievement over time

-   Collect and report other program performance data to meet GPRA
    requirements

# RESEARCH QUESTIONS

Research questions for this study include:

-   Do REA schools improve their reading achievement over time? Do REA
    schools show greater gains in reading than similar Title I schools?
    Do REA schools that serve large numbers of children who are English
    language learners show similar gains?

-   Do REA programs meet the criteria defined in the legislation at the
    district and school-level? What do REA programs look like? How are
    programs structured and what services do they provide? To what
    extent are all program components being implemented fully\-- are
    models composed of aligned professional development, tutoring, and
    family literacy components? Are districts providing adequate
    technical assistance and support?

-   In programs that are not well-implemented, what are the most
    significant barriers to implementation?

-   What are the program's effects on classroom instruction and teacher
    practice? What percentage of schools change reading instruction to
    align with research on effective practices?

-   How well are instructional strategies being implemented in schools
    and classrooms? How do schools choose reading materials and
    curricula? Are schools purchasing commercial materials and or
    models?

-   What is the extent of teacher and principal knowledge of
    research-based reading instruction and how does it change over time?

-   What is the content and process for professional development offered
    in REA sites---typical and ranges? Have schools changed the way they
    deliver professional development?

-   To what extent are schools' reading curriculums aligned with their
    state assessments?

-   What is the nature and quality of tutoring and family literacy
    components? How are they structured and how do they operate?

-   How is REA coordinated with other support in the schools for reading
    improvement (state initiatives, Even Start, Title I, CSRD, and
    others)?

# GENERAL EVALUATION DESIGN

The Framework for the Evaluation

This contractor shall develop a framework for evaluating K-3 reading
instruction. The framework shall enable a fact-based assessment of the
quality of three aspects of program implementation: professional
development, teacher knowledge, and instruction. The framework shall be
based on the criteria and principles laid out in the REA legislation,
and shall incorporate research-based principles of effective practice in
reading instruction as specified in the National Research Council\'s
*Preventing Reading Difficulties in Children,* and in NICHD research. It
shall also build on other design work supported by ED, particularly
design tasks supported by PES and REA. The implementation instruments to
be developed shall vary slightly in each year of the study. In the first
year of data collection it shall have the strongest focus on
professional development; in the second it shall have a greater emphasis
on teacher knowledge; and in the third a greater emphasis on
instructional practice.

The framework shall also include more general quality of implementation
indicators such as principal/administration support, provision of
professional development in line with best practices from research,
changes in materials, appropriate use of assessments in the curriculum,
technical assistance and other activities.

Relationship to Other REA Studies

Three new REA studies are being initiated this year\--the
State/District/School Performance System (performance system), School
and Classroom Implementation and Impact Study (SCII), and the
Children\'s Reading Gains (Gains study). These studies complement each
other and will be coordinated to maximize the power of the information
gained from each. Collectively, the three studies will answer questions
about what REA schools look like, how programs are implemented and
structured, and what outcomes they produce and for which students. While
the performance system will provide general descriptive information and
basic implementation information on all REA schools, the Implementation
study will answer questions about implementation and external validity.
Specifically, can the REA model be successfully implemented on a
large-scale basis and does it produce better outcomes in these schools
than typical reading instruction in Title I schools. The Gains study
focuses on questions of internal validity. Specifically, is the model
itself effective and for which students is it effective? How does it
change children\'s reading growth compared to typical Title I
instruction? This study will coordinate most closely with the Gains
study, particularly in the areas of sample selection, test
administration, and site visits.

The performance system will also serve as a sampling frame for the other
studies. The sample of 400 will be selected randomly from the universe
of schools in the performance system. The information gathered from the
performance system will help to identify adequately implemented schools
within the 400 from which to select the sample of 75 schools for the
Implementation Study, and will provide early information on the 75
schools from which to choose high-quality sites for the sample of 20
schools in the Gains Study.

### Relationship to Title I Studies {#relationship-to-title-i-studies .unnumbered}

The contractor for this study shall also collaborate with other
contractors who are conducting four complementary Title I studies that
will comprise the new National Assessment of Title I. The Implementation
contractor shall coordinate to the greatest extent with the contractor
conducting the Longitudinal Evaluation of the Effectiveness of School
Interventions (LEESI), and will administer the same assessment to
students in grades K-3 so the LEESI sample may be used as a comparison
group for this study. The Title I studies are intended to provide
information on the implementation and impact of the Title I program and
the standards-based reform efforts that Title I is designed to support.
Studies included in the new National Assessment of Title I are:

-   [Longitudinal Evaluation of the Effectiveness of School
    Interventions (LEESI)]{.underline}. This study will examine the
    impact of promising educational interventions on student
    achievement. The sample would consist of approximately 75
    high-poverty elementary schools operating Title I schoolwide
    programs, including programs receiving CSRD funds. The study will
    administer its own assessments in grades K-2 or grades 1-3, in order
    to obtain consistent assessment data across schools. In addition,
    the study would collect detailed data on school and classroom
    practices, including classroom observations as well as principal,
    teacher, and parent surveys. Data collection will occur in three
    school years: 2001-02, 2002-03, and 2003-04.

-   [National Study of Title I Schools]{.underline}. This study will be
    a longitudinal study of a nationally representative sample of Title
    I schools to examine their implementation of standards-based reform.
    The study will include surveys of principals, teachers, and teacher
    aides, analysis of state and district assessment data for the
    sampled schools, and case studies of 18 schools to obtain more
    in-depth information through interviews and document collection and
    analysis. The study will oversample high-poverty schools, schools
    identified for improvement, and schools with high concentrations of
    limited English proficient (LEP) students. Data collection will be
    conducted in three successive school years, beginning in the 2001-02
    school year.

-   [Evaluation of Title I Accountability Systems]{.underline}. This
    study will examine how states and districts identify schools for
    improvement and the strategies they employ to assist schools once
    they have been identified. The study will examine what kind of
    assistance schools receive as a result of being identified, whether
    that assistance results in change in school and classroom practices,
    and the extent to which student achievement improves. The sample
    would include 100 to 200 districts that have one or more schools
    identified as in need of improvement and a nested sample of
    identified schools within these districts. Data collection will
    include surveys of districts, principals, and teachers, as well as
    collection of state assessment data for the sampled schools. Data
    collection will occur in three school years: 2001-02, 2002-03, and
    2003-04.

-   [Comprehensive School Reform Demonstration Program: School
    Implementation Study]{.underline}. This study will examine the
    school-level implementation of the CSRD program and its support for
    research-based strategies for schoolwide reform and improvement.
    This study will be initiated a year after the other studies
    described above, and data collection will take place in 2002-03 and
    2003-04.

The Department intends that these related studies will be closely
coordinated with one another. Because all of the above studies include
school-level surveys, the Department anticipates that survey modules
focusing on specific issues may be included in more than one study. The
contractors for each study will be required to work with each other to
ensure consistent design of data collection instruments where
appropriate. The studies will also be coordinated in their methodologies
for analyzing state assessment data and other overlapping methodological
issues.

Design Elements

The contractor shall develop a thorough and complete design for the
project, which shall guide the evaluation. The design shall include a
description of the sampling methods, data collection plan, and data
analysis plan including a description of the methods for collecting and
analyzing state assessment data from the 400 sample schools, and
conducting an independent assessment in a sample of approximately 75
schools.

*Sampling Plan*

The sampling plan shall specify the numbers of respondents from which
data shall be collected, the basis for the sample size proposed, and the
calculations employed to determine the numbers proposed. The contractor
shall develop a sample design that maximizes the generalizability of
REA-funded schools, while oversampling schools that serve high numbers
of Limited English Proficient (LEP) students. A random sample of 400 REA
schools is proposed, and it shall be selected from among the universe of
REA schools identified in the REA performance system. Within these 400
schools, the contractor shall propose a strategy for selecting
principals, district administrators, and teachers to survey. The
contractor shall consider how to treat the two cohorts of REA awardees
in the sample design. The sample selection plan shall also describe how
a smaller sample of approximately 75 schools shall be selected for more
in-depth study and assessment.

*Data Collection Plan*

The contractor shall describe the data collection plan, including the
overall methodology, the types of data collection instruments that the
contractor will use, the specific data that the contractor shall
collect, and how the data will address each of the research questions.

The contractor shall describe the data collection methods for collecting
implementation data from approximately 400 schools beginning in February
of 2002 and continuing each winter/spring through 2004. The data
collection plan shall describe the focus of the instruments for each
survey year and justify that focus. In addition, different instruments
may be needed for the different grades, since their instructional goals
should be different. The contractor shall describe the different
emphases of data collection instruments for different grades.

The contractor shall describe the data collection methods for the
in-depth study in a subsample of approximately 75 schools in order to
validate the survey findings, to better understand survey responses, and
to gain more information on particular survey questions. The contractor
shall propose a design for the in-depth study that may include
interviews with teachers, principals, and administrators, review of
administrative records, and classroom observations.

*Assessment Data*

The contractor shall describe the methods for collecting and analyzing
available state assessment data on students served by REA in the 400
schools surveyed. The contractor shall also describe methods for
conducting an independent assessment to measure students' reading growth
in the 75 schools selected for the in-depth sub study. The study design
shall propose several options for assessments to be used and shall
describe the grades to be tested, the sampling strategies to be used to
select students, and how the achievement data shall be analyzed and
reported. Possibilities include assessing successive cohorts of first
graders, following a cohort of kindergartners through third grade, or
testing a sample each year that includes K-3 students. The contractor
shall consider how reference groups may be established for the REA
students\' achievement. In addition to using the sample in the LEESI as
a reference group, a nationally normed test would be particularly
beneficial for comparisons.

*Data Analysis Plan*

The contractor shall include a data analysis plan that describes how the
contractor proposes to analyze the data obtained from all data
collection instruments and assessments. The data analysis plan shall
include strategies for linking with an ongoing ED-funded task that will
collect state assessment data by school and longitudinal, individual
student data for some states, and for the analysis of that data for REA
schools in the large sample.

# PERIOD OF PERFORMANCE

The period of performance is 60 months.

# SCOPE OF WORK

The contractor shall conduct a comprehensive evaluation of the
implementation of the REA program at the school level and classroom
level, as well as an assessment of the impact of the program in a sample
of schools. The evaluation of implementation shall be based on the
criteria and principles required by the program's authorizing
legislation, and the quality of reading instruction in REA classrooms
shall be assessed based on research-based principles of effective
reading instruction as outlined in the National Research Council\'s,
*Preventing Reading Difficulties in Young Children*. The contractor
shall collect three years of program implementation and four years of
student assessment data, and prepare three reports, one after the first
year of implementation data collection, one after the second year of
implementation data collection, and a final report synthesizing the
findings from all study years. The contractor shall also provide a
minimum of two briefings a year to ED staff and other REA stakeholders
on the report findings.

# SUBTASKS

All deliverables shall be submitted electronically and in hard copy when
requested. The contractor shall also deliver an electronic copy of all
final deliverables to the contracting officer. Unless otherwise noted,
the COTR shall provide comments on all deliverables within two weeks of
receiving them and the contractor shall revise the deliverable and
re-submit it within two weeks of receiving the COTR's comments.

# TASK 1: MEETINGS WITH ED AND OTHER RELEVANT GROUPS {#task-1-meetings-with-ed-and-other-relevant-groups .unnumbered}

## Subtask 1.1 Meet with the Contracting Officer\'s Technical Representative (COTR) and Other Appropriate ED Staff {#subtask-1.1-meet-with-the-contracting-officers-technical-representative-cotr-and-other-appropriate-ed-staff .unnumbered}

The contractor shall meet with the contracting Officer\'s Technical
Representative (COTR), the contract Specialist (CS) and REA program
staff in Washington, D.C. within two weeks after the effective date of
the contract. The purpose of the meeting will be to discuss study design
and scheduling issues.

The contractor shall prepare a summary of the meeting, including a list
of next steps, within one week after the meeting. After approval by ED,
the contractor shall incorporate the minutes of this meeting and all
subsequent meetings as an element of regular project reports and into
the on-line library when it is set up as described in subtask 2.1.

## Subtask 1.2 Meet with Other Contractors {#subtask-1.2-meet-with-other-contractors .unnumbered}

The contractor shall meet, when requested by ED, with ED staff and other
contractors conducting related work for ED in order to coordinate
studies. The contractor shall participate in a minimum of two meetings
per year. Coordination topics shall include, but not be limited to: each
study\'s research and policy questions, information products, the
conceptual framework, study instruments, independent assessments, and
preliminary and final analyses and findings. The contractor shall
develop a coordination plan six weeks after the effective date of the
contract. These meetings will be scheduled by ED, shall last one day,
may be coordinated with REA Advisory Group meetings, and shall be held
in Washington, D.C. The contractor shall prepare minutes of each meeting
one week after each meeting is held.

## Subtask 1.3 Meet with REA Advisory Group (REAAG) {#subtask-1.3-meet-with-rea-advisory-group-reaag .unnumbered}

ED will convene REAAG meetings of seven to ten people. The group will be
selected by ED before the SCII contract is awarded and shall advise on
all three REA studies. The experts in this group will have extensive
experience evaluating reading instruction and or conducting research on
how children learn to read. The contractor in charge of the group will
be responsible for submitting a schedule of meetings, meeting agendas,
and meeting minutes to the COTR; but the contractor for this evaluation
shall be responsible for proposing agenda items and preparing briefing
materials that are related to their evaluation. During the course of the
contract, the contractor for the SCII shall participate in the advisory
group for no more than three meetings of two day's duration each project
year. Department staff will set the agenda for meetings and observe and
participate at these meetings.

The advisory group will provide the contractor with outside expertise on
the conduct of the evaluation including, but not limited to: refinements
of the evaluation design; data collection and instruments; analysis
plans; and the content and format of evaluation reports. The contractor
is free to accept or reject any advice or recommendations individual
panel members offer.

The contractors shall participate in the first meeting of the advisory
group within six weeks after the effective date of the contract.

**TASK 2 DEVELOP COMMUNICATION PLAN**

## Subtask 2.1 Establish and Operate On-Line Library for the Evaluation {#subtask-2.1-establish-and-operate-on-line-library-for-the-evaluation .unnumbered}

The contractor shall establish an on-line library no later than eight
weeks after the effective date of the award. The contractor shall
maintain copies of all minutes, reports, survey instruments,
administrative reports, the contractor Performance and Measurement
System information, REA Advisory Group meeting minutes, and other
contract-related documentation in a secure on-line library that is
accessible only to the COTR and key project staff. The contractor shall
put all documents approved by ED on the on-line library no later than
one week after their approval by ED.

## Subtask 2.2 Develop a Dissemination Plan {#subtask-2.2-develop-a-dissemination-plan .unnumbered}

The contractor shall develop a dissemination plan that reflects the
legislative mandate

and the purpose of information products that need to be developed and
disseminated to the appropriate audiences. These products shall use
plain, non-technical language to maximize their use. Identifying
audiences early on in the contract shall ensure that the study products
and reports are put to effective use. The contractor shall address the
following points in the dissemination plan:

-   identification of key study issues and appropriate audiences for
    dissemination of study findings

```{=html}
<!-- -->
```
-   types of media needed to get the information out to the appropriate
    audiences (print, video, CD ROM, Internet, satellite)

```{=html}
<!-- -->
```
-   estimated cost of developing and disseminating the information
    products which best meet the needs for this contract (brochures,
    interim reports, final report, related publications, alternate
    formats, Internet, listservs, satellite technology, electronic
    teleconferences, video and audio news releases in coordination with
    ED\'s Office of Public Affairs)

```{=html}
<!-- -->
```
-   suggested ways to evaluate product(s) effectiveness (surveys,
    response cards)

```{=html}
<!-- -->
```
-   tentative timetable for the optimal release of products (the COTR
    will provide information on Department initiatives, reauthorization
    timelines and ED "Calendar of Events" that may have implications for
    selecting a release date)

The contractor shall submit to the COTR an initial dissemination plan 17
weeks after the effective date of the contract. After a three-week ED
review, the contractor shall prepare a final plan 22 weeks after the
effective date of the contract.

## Subtask 2.3 Make presentations for ED staff and at professional and practitioner conferences {#subtask-2.3-make-presentations-for-ed-staff-and-at-professional-and-practitioner-conferences .unnumbered}

Throughout the contract, the contractor shall make presentations to ED
staff, and at professional and/or practitioner conferences, ranging
between one and three presentations per year. The contractor shall make
several presentations in the late spring and summer of 2005, after the
final report has been released.

To prepare for these presentations, the contractor shall develop
briefing materials that are non-technical and appropriate for the
general public. The contractor shall obtain the information on proposal
requirements and deadlines from each professional and/or practitioner
organization. The contractor shall provide a draft of the proposal and
briefing materials to the COTR at least one week before the submission
to each organization/conference planner, and make changes if requested
by the COTR. Examples of appropriate conferences include but are not
limited to:

REA State Directors Meetings

American Educational Research Association

Improving America's Schools Act Conferences

For each presentation the contractor shall submit the material for
presentation to the COTR for approval. The contractor **shall not**
present evaluation findings from reports or tabulations that have not
been **reviewed** and released by the Department. Prior to transmittal
of the final report to Congress, the contractor shall present only
methodology at any conferences or other public presentations.

# TASK 3 REFINE THE BASELINE MANAGEMENT PLAN  {#task-3-refine-the-baseline-management-plan .unnumbered}

The contractor shall submit a revised baseline management plan to the
COTR five weeks after the effective date of the contract. The plan shall
be revised from that submitted in the proposal to reflect topics
discussed in the initial meeting with ED. The contractor shall include
in the refined plan critical path diagrams, GANTT or PERT charts,
including person-loading charts by task. After the first year, the
contractor shall refine and update the plan for subsequent years to
incorporate refinements as needed.

# TASK 4 EVALUATION DESIGN AND ASSESSMENT PLAN {#task-4-evaluation-design-and-assessment-plan .unnumbered}

## Subtask 4.1 Refine evaluation design {#subtask-4.1-refine-evaluation-design .unnumbered}

The contractor shall refine the evaluation design and data collection
plan described in its proposal based on the initial meeting with ED
staff and REA Advisory Group suggestions. The contractor shall include
in the evaluation design all elements outlined under Design Elements in
the "General Evaluation Design" section above. The contractor shall
refine the data collection plan that clearly and succinctly describes
the sampling methodology, the data the contractor shall collect, how the
data shall be collected, the types of instruments to be used, and how
each data element shall answer one or more of the research questions.

## Subtask 4.2 Assessment plan {#subtask-4.2-assessment-plan .unnumbered}

The contractor shall review instruments to use for the independent
assessment of reading growth. The contractor shall review the
instruments being used by REA projects, in addition to those used
throughout states. The contractor shall select an instrument after a
thorough review has been completed, including an assessment of what
reference groups or comparisons could be employed using particular
assessments. The contractor shall then include an assessment strategy,
including sampling methods and grades to be tested, in the general
evaluation design.

The contractor shall submit the draft evaluation design and data
collection plan, specifying the assessment to be used for measuring
students' reading growth to the COTR within eight weeks after the
effective date of the contract. Allowing two weeks for Department review
and comment, the contractor shall submit the revised evaluation design
to the COTR within 12 weeks after the effective date of the contract.

## Subtask 4.3 Short OMB Package {#subtask-4.3-short-omb-package .unnumbered}

The contractor shall prepare a short version of the OMB clearance
package to request approval to implement the sampling plan and the first
year of assessment, in spring 2001. The contractor shall develop a forms
clearance package which focus on these activities while providing a
brief overview and context of the rest of the study. The package shall
be developed for the Department to submit to OMB under procedures of the
Paperwork Reduction Act and 5 CFR 1320. The contractor shall submit the
draft forms clearance package to the COTR within 10 weeks after the
effective date of the contract. Allowing two weeks for ED review and
comment, the contractor shall submit the revised forms clearance package
to the COTR within 14 weeks after the effective date of the contract.
The contractor shall request a 90-day approval for this package since it
is brief and straightforward. Allowing three months for OMB clearance,
the first assessment should be conducted in April and May of 2001.

# Task 5 SELECT AND NOTIFY SAMPLE AND RELEVANT ORGANIZATIONS {#task-5-select-and-notify-sample-and-relevant-organizations .unnumbered}

## Subtask 5.1 Prepare notification materials and school incentive strategy for sample {#subtask-5.1-prepare-notification-materials-and-school-incentive-strategy-for-sample .unnumbered}

The contractor shall prepare notification letters and information
packets for the participating Chief State School Officers (with a copy
to all identified state contacts for REA programs), and school
superintendents and appropriate school personnel in the local
educational agencies and schools included in the sample. The contractor
shall include in the letters and information packets general information
on the SCII as well as specific information on the data collection
schedule and plans, a discussion of the importance of the study, its
purposes, products, scheduled data collection and sample, provisions for
maintaining anonymity of survey participants, data security, the
organizations and persons involved in the study, and the benefits to be
derived from the study. The contractor shall also send a copy of this
letter to the Education Information Advisory Committee (EIAC)
representatives.

The contractor shall also propose a school incentive strategy for the 75
schools that shall participate in the in-depth study. This strategy
shall describe the amount and kind of compensation to be provided to
these schools. Once it has been approved, the incentive strategy shall
be described in the notification packets for the 75 schools
participating in the in-depth study only.

The contractor shall submit draft letters and all other notification
materials to ED no later than 18 weeks after the effective date of the
award. After a one-week review by ED, the contractor shall revise the
information packets as needed, print sufficient copies, and mail the
packets no later than 24 weeks after the effective date of the contract.

## Subtask 5.2 Select Sample {#subtask-5.2-select-sample .unnumbered}

The contractor shall select a random sample of approximately 400 schools
from within the universe of REA schools, as described in the sample
selection plan. The contractor shall then include the districts in which
the schools are located in the sample. The contractor shall then select
the teachers, principals, and administrators to be surveyed within the
sample of 400, as described in the sample selection plan. The contractor
shall then select the subsample of approximately 75 adequately
implemented schools for in-depth study and yearly assessments, and work
with the contractor for the Gains study to select the sample of 20
schools for their study. The contractor shall then select the
respondents within the 75 schools to participate in the site visits and
assessments. The contractor shall make initial contacts with all sites
to secure their participation.

The sample shall be selected no later than 20 weeks after the effective
date of the contract and contact shall be made with all sites no later
than 24 weeks after the effective date of the contract.

#  {#section .unnumbered}

# TASK 6 FIRST ASSESSMENT {#task-6-first-assessment .unnumbered}

## Subtask 6.1 Administer assessment {#subtask-6.1-administer-assessment .unnumbered}

The contractor shall administer the assessment selected in Subtask 4.2
to the subsample of students in the 75 schools selected in Subtask 5.2.
The assessment will be started by April and completed by the end of May
of 2001 (29 weeks-37 weeks after the effective date of the contract), in
accordance with the methodology and instruments approved by OMB.

##  {#section-1 .unnumbered}

## Subtask 6.2 Analyze and report data from 2001 assessment {#subtask-6.2-analyze-and-report-data-from-2001-assessment .unnumbered}

The contractor shall analyze the assessment data in accordance with the
final data analysis plan. The contractor shall submit to the COTR a
brief analysis and description of student performance 41 weeks after the
effective date of the contract.

# TASK 7 DATA COLLECTION INSTRUMENTS {#task-7-data-collection-instruments .unnumbered}

## Subtask 7.1 Develop data collection instruments {#subtask-7.1-develop-data-collection-instruments .unnumbered}

The contractor shall develop or select data collection instruments that
include all questions that the contractor shall ask of all respondents
for the evaluation. The contractor shall review relevant design papers
and instruments for other ED studies when developing data collection
instruments. The contractor shall incorporate instruments developed for
other ED studies as appropriate. The contractor shall submit the draft
data collection instruments within 28 weeks after the effective date of
the contract. Allowing three weeks for Department review and comment,
the contractor shall submit the revised data collection instruments to
the COTR within 33 weeks after the effective date of the contract.

## Subtask 7.2 Pilot test data collection instruments {#subtask-7.2-pilot-test-data-collection-instruments .unnumbered}

The contractor shall conduct a pilot test of the data collection
instruments developed and revised under Subtask 6.2. This pilot test
shall comply with Office of Management and Budget (OMB) requirements not
to ask the same questions of more than nine individuals. The contractor
shall select pilot test entities based on discussions with the COTR. The
contractor shall administer the pilot test in May and June of 2001
(beginning within 34 weeks after the effective date of the contract and
ending by 39 weeks after the effective date of the contract). Based on
feedback from the pilot test, the contractor shall make appropriate
revisions, finalize the data collection instruments, and submit them to
the COTR within 42 weeks after the effective date of the contract.

# TASK 8 COMPREHENSIVE OMB CLEARANCE PACKAGE  {#task-8-comprehensive-omb-clearance-package .unnumbered}

The contractor shall develop a forms clearance package for the SCII
which shall cover the next four years of the evaluation's data
collections. The package shall be developed for the Department to submit
to OMB under procedures of the Paperwork Reduction Act and 5 CFR 1320.
The contractor shall submit the draft forms clearance package to the
COTR within 45 weeks after the effective date of the contract. Allowing
three weeks for ED review and comment, the contractor shall submit the
revised forms clearance package to the COTR within 50 weeks after the
effective date of the contract.

The contractor shall schedule 5 months for both the review of the
clearance package by the Department's Information Management Team (IMT)
and by OMB prior to OMB approval. IMT or OMB may require revisions to
parts of the clearance package prior to approval. The contractor shall
make the required revisions and work with the COTR to respond to
questions from OMB and the public upon request during the OMB clearance
process (from approximately 51 weeks after the effective date of the
contract to approximately 71 weeks after the effective date of the
contract). The contractor shall, if necessary, meet with ED and OMB
staff to discuss the clearance package and its revisions and provide
other support for the clearance process. Allowing 5 months, OMB
clearance is expected to be received within 71 weeks after the effective
date of the contract (approximately at the end of January 2002).

### TASK 9 REFINE ANALYSIS PLAN {#task-9-refine-analysis-plan .unnumbered}

The contractor shall refine the preliminary analysis plan submitted with
its proposal. In the analysis plan, the contractor shall include a
detailed description of how the contractor shall analyze the data over
the course of the evaluation, including the techniques to analyze the
quantitative and qualitative data elements for each data collection
activity/instrument in the evaluation. The contractor shall include in
the analysis plan a description of all key variables to be analyzed, how
the data analysis addresses the research questions, and the types of
descriptive and inferential statistics that the contractor shall use,
along with table shells to illustrate planned analyses for the three
annual reports.

## Subtask 9.1 Linking with state assessment data bases and analysis of state assessment data {#subtask-9.1-linking-with-state-assessment-data-bases-and-analysis-of-state-assessment-data .unnumbered}

The analysis plan shall describe how this study will link with a
database of school-level and individual-level state assessment data
collected by another ED-funded task, to obtain data on REA schools in
the large sample. The plan shall describe how it will analyze both the
school-level and individual student data. Possible analytic strategies
for the school level data include analysis of a subsample of schools
that have assessment data in the early grades or analyzing data on third
or fourth grade students in the sample schools over a period of three or
more years. The contractor shall also propose strategies for conducting
additional analyses of data from states that collect individual
student-level data. These REA states are Texas, Alabama, Florida, and
Louisiana.

The contractor shall submit the revised analysis plan to the COTR no
later than 54 weeks after the effective date of the contract. Allowing
three weeks for ED review and comment, the contractor shall submit the
final analysis plan to the COTR no later than 60 weeks after the
effective date of the contract.

#  {#section-2 .unnumbered}

# TASK 10 2002 DATA COLLECTION, ANALYSIS, AND REPORTING {#task-10-2002-data-collection-analysis-and-reporting .unnumbered}

## Subtask 10.1 Collect data {#subtask-10.1-collect-data .unnumbered}

The contractor shall make final changes to the data collection
instruments and begin to administer the data collection instruments to
respondents within 72 weeks after the effective date of the contract or
within two weeks of receiving OMB clearance. The contractor shall begin
the 2002 data collection by February 1, 2002, and shall complete the
data collection by March 31, 2002. The contractor shall utilize
appropriate techniques to obtain response rates of at least 85 percent
(e.g., postcard reminders and telephone follow up).

## Subtask 10.2 Student standardized test administration {#subtask-10.2-student-standardized-test-administration .unnumbered}

The assessment will be administered in March and April of 2002 (from 76
weeks to 80 weeks after the effective date of the contract), in
accordance with the methodology and instruments approved by OMB.

##  {#section-3 .unnumbered}

## Subtask 10.3 Analyze and process 2002 data {#subtask-10.3-analyze-and-process-2002-data .unnumbered}

The contractor shall develop coding materials for entering the data
collected and preparing the data for analysis as it is received. To
ensure accuracy, the contractor shall verify all key data entered,
conduct edit and consistency checks, and track response rates. The
contractor shall resolve problems identified in this process through
phone calls to the respondents. The contractor shall include information
on the status of this task in each monthly progress report.

The contractor shall analyze the data received from the surveys in
accordance with the final data analysis plan. The contractor shall
submit to the COTR preliminary tables based on tabulations from the
initial data analysis, as well as a brief description of initial
findings from these analyses within 88 weeks of the effective date of
the contract. (Findings from the second independent assessment may be
included if feasible, but are not required at this time).

## Subtask 10.4 Prepare first report {#subtask-10.4-prepare-first-report .unnumbered}

The contractor shall prepare the first report and a non-technical
executive summary summarizing the first two years of data collection
(two years of assessment data and one year of implementation data) from
the SCII. The contractor shall submit an outline for approval, 92 weeks
after the effective date of the contract, before beginning work on the
report. The report shall include descriptive and analytic information
that addresses the research questions and analytic charts, graphs, and
tables to summarize the data on school and district characteristics and
program services. The report shall also address other issues as agreed
upon by the COTR in any subsequent meetings or correspondence, and that
describes the level and quality of implementation of REA programs, and
the quality of reading instruction in REA classrooms.

The contractor shall submit a draft of the first SCII report including
an executive summary to the COTR no later than 100 weeks after the
effective date of the contract. This draft shall include findings from
the second independent assessment. After a three-week review by the
Department, the contractor shall submit a second draft to the COTR no
later than 106 weeks after the effective date of the contract. After a
three-week review by the Department, the contractor shall submit a third
draft to the COTR no later than 111 weeks after the effective date of
the contract.

##  {#section-4 .unnumbered}

## Subtask 10.5 Disseminate to REA Advisory Group (REAAG) {#subtask-10.5-disseminate-to-rea-advisory-group-reaag .unnumbered}

The contractor shall disseminate the second draft of the first report to
the REAAG for comment no later than 111 weeks after the effective date
of the contract. After a three-week review by the REAAG and Department,
the contractor shall incorporate final changes, and submit the final
report to the COTR no later than 116 weeks after the effective date of
the contract. The COTR will transmit the year one report to Congress
after receiving approval from senior Department officials.

## Subtask 10.6 Disseminate first SCII report {#subtask-10.6-disseminate-first-scii-report .unnumbered}

The contractor shall disseminate the first SCII report, as specified in
the dissemination plan, two weeks after it is transmitted to Congress.

### TASK 11 2003 DATA COLLECTION, ANALYSIS, AND REPORTING {#task-11-2003-data-collection-analysis-and-reporting .unnumbered}

## Subtask 11.1 Collect 2003 data {#subtask-11.1-collect-2003-data .unnumbered}

The contractor shall make any minor changes to the data collection
instruments based on experience and information gathered in the first
data collection and resubmit the instruments to the COTR within 119
weeks after the effective date of the contract, by January 4, 2003.
After receiving ED comments, and revising the instruments, the
contractor shall begin to administer the data collection instruments to
respondents within 123 weeks of the effective date of the contract (by
approximately February 1, 2003).

The contractor shall complete the second year of data collection within
131 weeks of the effective date of the contract (by approximately March
31, 2003). The contractor shall utilize appropriate techniques to obtain
response rates of at least 85 percent (e.g., postcard reminders and
telephone follow up).

## Subtask 11.2 Student standardized test administration {#subtask-11.2-student-standardized-test-administration .unnumbered}

The assessment will be administered in March and April of 2003, in
accordance with the methodology and instruments approved by OMB.

##  {#section-5 .unnumbered}

## Subtask 11.3 Analyze and process second year of data {#subtask-11.3-analyze-and-process-second-year-of-data .unnumbered}

The contractor shall develop coding materials for entering the data
collected and preparing the data for analysis as it is received. To
ensure accuracy, the contractor shall verify all key data entered,
conduct edit and consistency checks, and track response rates. The
contractor shall resolve problems identified in this process through
phone calls to the respondents. The contractor shall include information
on the status of this task in each monthly progress report.

The contractor shall analyze the data received from the surveys in
accordance with the final data analysis plan. The contractor shall
submit to the COTR preliminary tables based on tabulations from the
initial data analysis, as well as a brief description of initial
findings from these analyses within 139 weeks after the effective date
of the contract. (Findings from the third independent assessment may be
included if feasible, but are not required at this time).

## Subtask 11.4 Prepare second SCII report {#subtask-11.4-prepare-second-scii-report .unnumbered}

The contractor shall prepare the second report and a non-technical
executive summary summarizing the findings of the three test
administrations and the two years of implementation data collected. The
contractor shall include in the report descriptive and analytic
information that addresses the research questions outlined above and as
agreed upon by the COTR in any subsequent meetings or correspondence.
The contractor shall incorporate into the final second report
information and findings from not more than five other studies or
databases conducted or maintained for the Department or by independent
researchers where they are relevant to the research questions for this
evaluation, as well as trends and patterns observed from year one to
year two.

The contractor shall submit an outline for the second report within 143
weeks after the effective date of the contract. This report shall
include findings from the second independent assessment. After a review
and comments from the COTR, the contractor shall submit a draft to the
COTR no later than 151 weeks after the effective date of the contract.
After a three-week review by the Department, the contractor shall submit
a second draft to the COTR no later than 157 weeks after the effective
date of the contract. After a three-week review by the Department, the
contractor shall submit a third draft to the COTR no later than 162
weeks after the effective date of the contract.

## Subtask 11.5 Disseminate to REA Advisory Group (REAAG) {#subtask-11.5-disseminate-to-rea-advisory-group-reaag .unnumbered}

The contractor shall disseminate the third draft of the second SCII
report to the REAAG no later than 162 weeks after the effective date of
the contract. After a three-week review by the REAAG and Department, the
contractor shall submit the final report to the COTR no later than 167
weeks after the effective date of the contract. The COTR will transmit
the second report to Congress after receiving approval from senior
Department officials.

##  {#section-6 .unnumbered}

## Subtask 11.6 Disseminate second SCII report {#subtask-11.6-disseminate-second-scii-report .unnumbered}

# The contractor shall disseminate the second SCII report, as specified in the dissemination plan, two weeks after it is transmitted to Congress. {#the-contractor-shall-disseminate-the-second-scii-report-as-specified-in-the-dissemination-plan-two-weeks-after-it-is-transmitted-to-congress. .unnumbered}

#  {#section-7 .unnumbered}

# TASK 12 2004 DATA COLLECTION, ANALYSIS AND REPORTING {#task-12-2004-data-collection-analysis-and-reporting .unnumbered}

## Subtask 12.1 Collect 2004 data {#subtask-12.1-collect-2004-data .unnumbered}

The contractor shall make any minor changes to the data collection
instruments based on experience and information gained from the 2003
data collection, and submit the revised instruments to the COTR 170
weeks after the effective date of the contract. The COTR will provide
feedback or final approval, and the contractor shall begin to the final
data collection by the beginning of February, 2004 and complete it by
the end of March (from 175 to 183 weeks after the effective date of the
contract). The contractor shall utilize appropriate techniques to obtain
response rates of at least 85 percent (e.g., postcard reminders and
telephone follow up).

## Subtask 12.2 Student standardized test administration {#subtask-12.2-student-standardized-test-administration .unnumbered}

The assessment will be administered in March and April of 2004, in
accordance with the methodology and instruments approved by OMB.

## Subtask 12.3 Analyze and process data  {#subtask-12.3-analyze-and-process-data .unnumbered}

The contractor shall develop coding materials for entering the data
collected and preparing the data for analysis as it is received. To
ensure accuracy, the contractor shall verify all key data entered,
conduct edit and consistency checks, and track response rates. The
contractor shall include information on the status of this task in each
monthly progress report.

The contractor shall analyze the data received from the surveys in
accordance with the final data analysis plan. The contractor shall
submit to the COTR preliminary tables based on tabulations from the
initial data analysis, as well as a brief description of initial
findings from these analyses within 191 weeks after the effective date
of the contract. (Findings from the fourth independent assessment may be
included if feasible, but are not required at this time).

## Subtask 12.4 Prepare final SCII report {#subtask-12.4-prepare-final-scii-report .unnumbered}

The contractor shall prepare an outline for the final SCII report within
195 weeks after the effective date of the contract. After a two-week
review by the COTR, the contractor shall submit a second draft of the
report outline within 199 weeks of the effective date of the contract.
After a two-week review of the outline by ED staff, the contractor shall
submit a draft of the final report including an executive summary
summarizing the findings of the evaluation within 210 weeks after the
effective date of the contract. This report shall include findings from
the fourth independent assessment. The contractor shall include in the
report descriptive and analytic information that addresses the research
questions outlined above and as agreed upon by the COTR in any
subsequent meetings or correspondence.

The contractor shall incorporate into the final SCII report information
and findings from the prior years of the study, and an analysis of how
the implementation of REA has changed over time in the sample schools.
After a three-week review by ED staff, the contractor shall submit a
second draft to the COTR no later than 216 weeks after the effective
date of the contract. After a three-week review by the Department, the
contractor shall submit a third draft to the COTR and the REAAG no later
than 222 weeks after the effective date of the contract. After a
three-week review by the Department and the REAAG and final revisions,
the contractor shall submit the final report to the COTR no later than
229 weeks after the effective date of the contract. The COTR will
transmit the report to Congress after receiving approval from senior
Department officials.

## Subtask 12.5 Disseminate Final Report {#subtask-12.5-disseminate-final-report .unnumbered}

The contractor shall disseminate the final SCII report, as specified in
the dissemination plan, two weeks after the report is disseminated to
Congress.

**TASK 13 ARCHIVING OF DATA**

## Subtask 13.1 Public Use Data Files {#subtask-13.1-public-use-data-files .unnumbered}

The contractor shall prepare public use data files that are formatted to
the National Center for Education Statistics (NCES) Electronic Codebook
(ECB).

All databases developed or modified for the Department of Education will
be modeled in an entity relation (E-R) diagram model for evaluation by
ED for acceptability for compatibility with other ED databases. The
criteria for normalization which will be evaluated by third party IV&V
will be at least 3rd normal form. Acceptable modeling tools for
delivering soft copy of E-R models include ERWin and Oracle Designer
2000 but will be specifically defined by the COTR. The standard
enterprise database environment in use by the Department of Education is
MicroSoft SQL/Server 7.0. Access may be used for very small efforts with
little probability of growth, Oracle may be used for very large, high
transaction efforts. The use of Access or Oracle needs to be
specifically approved by the Office of the Chief Information Officer.

## Subtask 13.2 Transmitting the Data Files to ED {#subtask-13.2-transmitting-the-data-files-to-ed .unnumbered}

Upon completion of each year of the evaluation and transmission of each
report to Congress, the contractor shall provide hard copy and
electronic medium copies of the data set, code books, technical reports
and other study materials to an archival site for public dissemination.
The contractor shall obtain the COTR's approval for the specific
archival site to be used. The contractor shall ensure that the archived
materials are in compliance with privacy protection laws. The electronic
data sets shall be submitted in ASCII format with documentation of field
names and widths. The contractor shall complete this task for year one
no later than 120 weeks after the effective date of the contract; for
year two no later than 171 weeks after the effective date of the
contract; and for year three no later than 233 weeks after the effective
date of the contract.

**TASK 14 CONTRACTOR MONTHLY REPORTING REQUIREMENTS**

The contractor shall establish an internal system with the capacity to:

Identify problem areas by order of importance;

Identify anticipated schedule slippage and cost overruns; and

Provide means of determining where project managers and resources are
deployed to assist more critical tasks. The contractor shall include
this information in the monthly progress reports. The progress report
shall include both monthly and cumulative contract costs by task, for
the full evaluation, and by four deliverables (draft evaluation design,
OMB clearance process, draft reports, and database) including an
assessment of whether each deliverable was completed at or under cost
upon submission of each deliverable.

The contractor shall provide documentation of how this system will work
to the COTR within one week of the effective date of the contract. The
contractor shall submit two copies of these reports to the COTR and
contracting officer one week after the end of each reporting month.

# VIII. Schedule of Deliverables {#viii.-schedule-of-deliverables .unnumbered}

  ------- ------------------------------------ -------------- --------------
  Task    Deliverable                          Due (weeks     Calendar Month
                                               after          (Dates are
                                               effective date approximate
                                               of contract)   and based on
                                                              an 8/31 award
                                                              date)

  1.1     Summary of meeting with COTR and ED  3 weeks        September 21,
          staff and next steps                                2000

  1.2     Minutes of meetings with ED and      1 week after   
          other contractors                    meeting is     
                                               held           

  3       Refine baseline management plan      5 weeks        October 7,
                                                              2000

  1.2     Coordination plan                    6 weeks        October 14,
                                                              2000

  1.3     First Advisory Group Meeting         6 weeks        October 14,
                                                              2000

  2.3     Presentation materials               1 week before  
                                               presentation   

  2.1     On-line library                      8 weeks        October 28,
                                                              2000

  4.1,2   Refine evaluation design (including  8 weeks        October 28,
          assessment plan)                                    2000

  4.3     Short OMB package                    10 weeks       November 11,
                                                              2000

  4.1,2   Final evaluation design              12 weeks       November 25,
                                                              2000

  4.3     Revised Short OMB package            14 weeks       December 9,
                                                              2000

  2.2     Draft dissemination plan             17 weeks       December 30,
                                                              2000

  5.1     Notification materials               18 weeks       January 9,
                                                              2001

  5.2     Select sample                        20 weeks       January 22,
                                                              2001

  2.2     Final dissemination plan             22 weeks       February 5,
                                                              2000

  5.1, 2  Contact sites and mail notification  24 weeks       February 19,
          materials                                           2001

  6.1     Administer first assessment          29 weeks-37    April --May
                                               weeks          2001

  7.1     Instruments                          28 weeks       March 29, 2001

  7.1     Revised Instruments                  33 weeks       May 4, 2001

  7.2     Conduct pilot test                   34 weeks-39    May 11-June
                                               weeks          14, 2001

  6.2     Data from first assessment           41 weeks       June 28, 2001

  7.2     Revised Instruments after pilot test 42 weeks       July 7, 2001

  8       OMB clearance package                45 weeks       July 28, 2001

  8       Revised OMB clearance package        50 weeks       August 31,
                                                              2001

  9       Revised analysis plan                54 weeks       September 28,
                                                              2001

  9       Final analysis plan                  60 weeks       November 8,
                                                              2001

          Forms Clearance Support during       51-71 weeks    September 7,
          process of OMB approval                             2001-January
                                                              25, 2002

          2002 Data Collection                 72 weeks-80    February
                                               weeks          1-March 31,
                                                              2002

          2002 Assessment                      76 weeks-84    March 1-April
                                               weeks          30, 2002

  10.3    Preliminary tabulations from 2002    88 weeks       May 30, 2002
          data                                                

  10.4    Draft outline of findings for first  92 weeks       June 30, 2002
          report                                              

  10.4    Draft of first report                100 weeks      August 20,
                                                              2002

  10.4    2^nd^ draft first report             106 weeks      September 30,
                                                              2002

  10.4    3^rd^ draft first report             111 weeks      November 7,
                                                              2002

  10.4    Final first report                   116 weeks      December 13,
                                                              2002

  11.1    Revised data collection instruments  119 weeks      January 4,
          for 2003                                            2003

  13.2    Public use data files for 2002 data  120 weeks      January 11,
                                                              2003

          2003 Data collection                 123 weeks-131  February
                                               weeks          1-March 31,
                                                              2003

          2003 Assessment                      127 weeks-135  March 1- April
                                               weeks          30, 2003

  11.3    Preliminary tabulations for year     139 weeks      May 30, 2003
          2003 data                                           

  11.4    Second report outline                143 weeks      June 28, 2003

  11.4    First draft of second report         151 weeks      August 19,
                                                              2003

  11.4    Second draft of second report        157 weeks      September 27,
                                                              2003

  11.4    Third draft of second report         162 weeks      November 1,
                                                              2003

  11.4    Final draft of second report         167 weeks      December 6,
                                                              2003

  13.2    Public use data files for 2003 data  171 weeks      December 13,
                                                              2003

  12.1    Revised instruments for 2004 data    170 weeks      December 27,
          collection                                          2003

          2004 Data collection                 175 weeks-183  February
                                               weeks          1-March 31,
                                                              2004

          2004 Assessment                      179 weeks-187  March 1-April
                                               weeks          30, 2004

  12.3    Preliminary tabulations for year     191 weeks      May 30, 2004
          2004 data                                           

  12.4    Final report outline                 195 weeks      June 30, 2004

  12.4    2^nd^ Draft outline                  199 weeks      July 28, 2004

  12.4    First draft of final report          210 weeks      October 11,
                                                              2004

  12.4    Second draft of final report         216 weeks      November 24,
                                                              2004

  12.4    Third draft of final report          222 weeks      January 6 ,
                                                              2005

  12.4    Final draft of final report          229 weeks      February 15,
                                                              2005

  13.2    Public use data files for 2004 data  233 weeks      March 23, 2005

  14      Monthly reports                      Every month,   Beginning
                                               one week after October 7,
                                               the end of     2000
                                               each month     
  ------- ------------------------------------ -------------- --------------