**EDUCATION RESEARCH GRANTS**

**CFDA NUMBER: 84.305**

**RELEASE DATE: April 6, 2007**

**REQUEST FOR APPLICATIONS NUMBER: IES-NCER-2008-01**

**INSTITUTE OF EDUCATION SCIENCES**

[**http://ies.ed.gov/**](http://ies.ed.gov/)

**LETTER OF INTENT RECEIPT DATE: May 24, 2007 and September 6, 2007**

**APPLICATION DEADLINE DATE: July 26, 2007 and November 1, 2007**

**THIS REQUEST FOR APPLICATIONS CONTAINS THE FOLLOWING INFORMATION:**

###### Section Page

[PART I. GENERAL OVERVIEW 3](#part-i.-general-overview)

[**1. REQUEST FOR APPLICATIONS** 3](#request-for-applications)

[**2. OVERVIEW OF THE INSTITUTE\'S RESEARCH PROGRAMS**
3](#overview-of-the-institutes-research-programs)

[*A. Outcomes* 4](#a.-outcomes)

[*B. Conditions* 4](#b.-conditions)

[*C. Grade Levels* 5](#__RefHeading___Toc164760577)

[*D. Research Goals* 5](#d.-research-goals)

[PART II. RESEARCH GRANT TOPICS 7](#part-ii.-research-grant-topics)

[**1. READING AND WRITING** 7](#reading-and-writing)

[*A. Purpose* 7](#a.-purpose)

[*B. Background* 7](#b.-background)

[*C. Specific Requirements* 8](#c.-specific-requirements)

[PART III. REQUIREMENTS OF THE PROPOSED RESEARCH
10](#part-iii.-requirements-of-the-proposed-research)

[**14. GENERAL REQUIREMENTS OF THE PROPOSED RESEARCH**
10](#general-requirements-of-the-proposed-research)

[*A. Basic Requirements* 10](#a.-basic-requirements)

[*B. Requirements for Goal One (Identification Projects)*
10](#__RefHeading___Toc164760587)

[*C. Requirements for Goal Two (Development Projects)*
13](#c.-requirements-for-goal-two-development-projects)

[*D. Requirements for Goal Three (Efficacy and Replication Trials)*
16](#d.-requirements-for-goal-three-efficacy-and-replication-trials)

[*E. Requirements for Goal Four (Scale-up Evaluations)*
22](#__RefHeading___Toc164760590)

[*F. Requirements for Goal Five (Measurement Projects for Topics 1-3,
6-7, 10-13)*
26](#f.-requirements-for-goal-five-measurement-projects-for-topics-1-3-6-7-10-13)

[PART IV. GENERAL SUBMISSION AND REVIEW INFORMATION
28](#part-iv.-general-submission-and-review-information)

[**15. APPLICATIONS AVAILABLE** 28](#applications-available)

[**16. MECHANISM OF SUPPORT** 28](#mechanism-of-support)

[**17. FUNDING AVAILABLE** 28](#__RefHeading___Toc164760595)

[**18. ELIGIBLE APPLICANTS** 28](#eligible-applicants)

[**19. SPECIAL REQUIREMENTS** 28](#special-requirements)

[**20. LETTER OF INTENT** 29](#letter-of-intent)

[**21. SUBMITTING AN APPLICATION** 30](#submitting-an-application)

[**22. CONTENTS OF APPLICATION** 30](#contents-of-application)

[*A. Project Summary/Abstract* 30](#a.-project-summaryabstract)

[*B. Project Narrative* 31](#b.-project-narrative)

[*C. Bibliography and References Cited*
32](#c.-bibliography-and-references-cited)

[*D. Biographical Sketches of Senior/Key Personnel*
32](#d.-biographical-sketches-of-seniorkey-personnel)

[*E. Narrative Budget Justification*
32](#e.-narrative-budget-justification)

[*F. Subaward Budget* 33](#f.-subaward-budget)

[*G. Appendix A* 33](#g.-appendix-a)

[*H. Appendix B (optional)* 33](#h.-appendix-b-optional)

[*I. Research on Human Subjects* 34](#i.-research-on-human-subjects)

[*J. Additional Forms* 34](#j.-additional-forms)

[**23. APPLICATION PROCESSING** 34](#application-processing)

[**24. PEER REVIEW PROCESS** 34](#peer-review-process)

[**25. REVIEW CRITERIA FOR SCIENTIFIC MERIT**
35](#review-criteria-for-scientific-merit)

[*A. Significance* 35](#a.-significance)

[*B. Research Plan* 35](#b.-research-plan)

[*C. Personnel* 35](#c.-personnel)

[*D. Resources* 35](#d.-resources)

[**26. RECEIPT AND START DATE SCHEDULE**
36](#receipt-and-start-date-schedule)

[**27. AWARD DECISIONS** 36](#__RefHeading___Toc164760619)

[**28. INQUIRIES MAY BE SENT TO** 36](#inquiries-may-be-sent-to)

[**29. PROGRAM AUTHORITY** 39](#program-authority)

[**30. APPLICABLE REGULATIONS** 39](#applicable-regulations)

[**31. REFERENCES** 39](#references)

# PART I. GENERAL OVERVIEW

## **[1. REQUEST FOR APPLICATIONS]{.underline}**

In this announcement, the Institute of Education Sciences (Institute)
describes the research grant programs that are funded through the
National Center for Education Research. Separate announcements are
available on the Institute\'s website that pertain to the predoctoral
and postdoctoral research training programs, and national research and
development centers funded through the National Center for Education
Research and to the discretionary grant competitions funded through the
Institute\'s National Center for Special Education Research
(<http://ies.ed.gov/ncser>).

The Institute invites applications for research projects that will
contribute to its education research programs in Reading and Writing;
Mathematics and Science Education; Cognition and Student Learning;
Teacher Quality -- Reading and Writing; Teacher Quality -- Mathematics
and Science Education; Social and Behavioral Context for Academic
Learning; Education Leadership; Education Policy, Finance, and Systems;
Early Childhood Programs and Policies; High School Reform; Interventions
for Struggling Adolescent and Adult Readers and Writers; Postsecondary
Education; and Education Technology. For the FY 2008 competition, the
Institute will consider only applications that meet the requirements
outlined below under [Part II Research Grant
Topics](#part-ii.-research-grant-topics) and [Part III Requirements for
the Proposed Research](#general-requirements-of-the-proposed-research).

## **[2. OVERVIEW OF THE INSTITUTE\'S RESEARCH PROGRAMS]{.underline}**

The Institute\'s over-arching priority is research that contributes to
improved academic achievement for all students, and particularly for
those whose education prospects are hindered by inadequate education
services and conditions associated with poverty, race/ethnicity, limited
English proficiency, disability, and family circumstance.

With academic achievement as the major priority, the Institute focuses
on outcomes that differ by periods of education. In the infancy and
preschool period, the outcomes of interest are those that enhance
readiness for schooling, for example, language skills, and for infants
and toddlers with disabilities, developmental outcomes. In kindergarten
through 12th grade, the core academic outcomes of reading and writing
(including reading and writing in the disciplines), mathematics, and
science are emphasized, as well as the behaviors and social skills that
support learning in school and successful transitions to employment,
independent living, and post-secondary education. At the post-secondary
level, the focus is on enrollment in and completion of programs that
prepare students for successful careers and lives. The same outcomes are
emphasized for students with disabilities across each of these periods,
and include the functional outcomes that improve educational and
transitional results. The acquisition of basic skills by adults with low
levels of education is also a priority.

In conducting research on academic outcomes, the Institute concentrates
on conditions within the control of the education system, with the aim
of identifying, developing, and validating effective education programs,
practices, policies, and approaches as well as understanding the factors
that influence variation in their effectiveness such as implementation.
Conditions that are of highest priority to the Institute are in the
areas of curriculum, instruction, assessment (including the
identification of students with disabilities), the quality of the
education workforce, and the systems and policies that affect these
conditions and their interrelationships (for example, accountability
systems, delivery mechanisms including technology, and policies that
support the ability of parents to improve educational results for their
children through such means as choice of education services and
provision of school-related learning opportunities in the home).

In this section, the Institute describes the overall framework for its
research grant programs. Specific information on the research topics
described in this announcement may be found in the sections pertaining
to each education research program:

-   Reading and Writing

-   Mathematics and Science Education

-   Cognition and Student Learning

-   Teacher Quality -- Reading and Writing

-   Teacher Quality -- Mathematics and Science Education

-   Social and Behavioral Context for Academic Learning

-   Education Leadership

-   Education Policy, Finance, and Systems

-   Early Childhood Programs and Policies

-   High School Reform

-   Interventions for Struggling Adolescent and Adult Readers and
    Writers

-   Postsecondary Education

-   Education Technology

The Institute addresses the educational needs of typically developing
students through its Education Research programs and the needs of
students with disabilities through its Special Education Research
programs. Both the Education Research and the Special Education Research
programs are organized by [outcomes](#a.-outcomes) (e.g., reading,
mathematics), [type of education condition](#b.-conditions) (e.g.,
curriculum and instruction; teacher quality; administration, systems,
and policy), [grade level](#PI_2C), and [research
goals](#d.-research-goals).

### A. Outcomes

The Institute\'s research programs focus on improvement of the following
education outcomes: (a) readiness for schooling (pre-reading,
pre-writing, early mathematics and science knowledge and skills, and
social development); (b) academic outcomes in reading, writing,
mathematics, and science; (c) student behavior and social interactions
within schools that affect the learning of academic content; (d) skills
that support independent living for students with significant
disabilities; and (e) educational attainment (high school graduation,
enrollment in and completion of post-secondary education).

### B. Conditions

In general, each of the Institute\'s research programs focuses on a
particular type of condition (e.g., curriculum and instruction) that may
affect one or more of the outcomes listed previously (e.g., reading).
The Institute\'s research programs are listed below according to the
primary condition that is the focus of the program.

**a. *Curriculum and instruction***. Several of the Institute\'s
programs focus on the development and evaluation of curricula and
instructional approaches. These programs include: (a) Reading and
Writing; (b) Mathematics and Science Education; (c) Cognition and
Student Learning; (d) Social and Behavioral Context for Academic
Learning; (e) Early Childhood Programs and Policies; (f) Interventions
for Struggling Adolescent and Adult Readers and Writers; and (g)
Education Technology.

**b**. ***Quality of the Education Workforce***. A second condition that
affects student learning and achievement is the quality of teachers and
education leaders (e.g., principals, superintendents). The Institute
funds research on how to improve teacher quality through its programs on
(a) Teacher Quality -- Reading and Writing; (b) Teacher Quality --
Mathematics and Science Education; and (c) Research on Education
Leadership.

**c. *Administration, systems, and policy***. A third approach to
improving student outcomes is to identify systemic changes in the ways
in which schools and districts are led, organized, managed, and operated
that may be directly or indirectly linked to student outcomes. The
Institute takes this approach in its programs on (a) Education Policy,
Finance, and Systems; (b) Early Childhood Programs and Policies; (c)
High School Reform; and (d) Postsecondary Education.

Applicants should be aware that some of the Institute\'s programs cover
multiple conditions. For example, the following programs cover multiple
conditions: (a) Cognition and Student Learning; (b) Early Childhood
Programs and Policies; (c) High School Reform; (d) Education Technology;
and (e) Postsecondary Education.

[]{#PI_2C .anchor}

### C. Grade Levels

The Institute\'s research programs also specify the ages or grade levels
covered in the research program. The specific grades vary across
research programs and within each research program, and grades may vary
across the research goals. In general, the Institute supports research
for (a) pre-kindergarten and kindergarten, (b) elementary school, (c)
middle school, (d) high school, (e) post-secondary education, (f)
vocational education, and (g) adult education. In addition, the
Institute supports research on infants with disabilities.

### D. Research Goals

The Institute has established five research goals for its research
programs. Within each research program one or more of the goals may
apply: (a) [Goal One](#PIII_B_Goal1) -- identify existing programs,
practices, and policies that may have an impact on student outcomes and
the factors that may mediate or moderate the effects of these programs,
practices, and policies; (b) [Goal Two](#PIII_C_Goal2) -- develop
programs, practices, and policies that are theoretically and empirically
based; (c) [Goal
Three](#d.-requirements-for-goal-three-efficacy-and-replication-trials)
-- establish the efficacy of fully developed programs, practices, and
policies; (d) [Goal Four](#PIII_E_Goal4) -- provide evidence on the
effectiveness of programs, practices, and policies implemented at scale;
and (e) [Goal
Five](#f.-requirements-for-goal-five-measurement-projects-for-topics-1-3-6-7-10-13)
-- develop or validate data and measurement systems and tools.

For a list of the Institute\'s FY 2008 research and training grant
topics -- including grant competitions through the Institute\'s National
Center for Education Research and National Center for Special Education
Research, please see Table 1 below. Funding announcements for these
competitions may be downloaded from the Institute\'s website at
[http://ies.ed.gov](http://ies.ed.gov/).

**Table 1: FY 2008 Research and Training Grant Topics**

**[National Center for Education Research]{.underline}**

1\. Research Grant Topics

-   Reading and Writing

-   Mathematics and Science Education

-   Cognition and Student Learning

-   Teacher Quality -- Reading and Writing

-   Teacher Quality -- Mathematics and Science Education

-   Social and Behavioral Context for Academic Learning

-   Education Leadership

-   Education Policy, Finance, and Systems

-   Early Childhood Programs and Policies

-   High School Reform

-   Interventions for Struggling Adolescent and Adult Readers and
    Writers

-   Postsecondary Education

-   Education Technology

2\. Research Training Grant Topics

-   Postdoctoral Research Training Program

-   Predoctoral Research Training Program

3\. National Research and Development Center Topics

-   Cognition and Science Instruction

-   Instructional Technology

**[National Center for Special Education Research]{.underline}**

1\. Research Grant Topics

-   Early Intervention, Early Childhood Special Education, and
    Assessment for Young Children with Disabilities Research

-   Mathematics and Science Special Education Research

-   Reading, Writing, and Language Development Special Education
    Research

-   Serious Behavior Disorders Special Education Research

-   Individualized Education Programs and Individualized Family Service
    Plans Research

-   Secondary and Transition Services Research

-   Autism Spectrum Disorders Research

-   Response to Intervention Research

-   Related Services Special Education Research

2\. Research Training Grant Topics

-   Postdoctoral Special Education Research Training

3\. National Research and Development Center Topics

-   Center on Serious Behavior Disorders at the Secondary Level

-   Center on Response to Intervention in Early Childhood Special
    Education

# PART II. RESEARCH GRANT TOPICS

For FY 2008, the Institute\'s National Center for Education Research is
accepting applications for research grants under 13 topics. There are
two application deadlines for each of these 13 topics: July 26, 2007,
and November 1, 2007. In this section, the Institute describes the
research grant topics.

Across its research programs, the National Center for Education Research
is particularly interested in interventions for students who are from
low income backgrounds and/or racial, ethnic, and linguistic minority
groups that have underachieved academically, but will consider
applications that focus on other populations if the results are likely
to be applicable across socioeconomic, racial, ethnic, and linguistic
categories.

## **[1. READING AND WRITING]{.underline}**

Program Officer: Dr. Elizabeth Albro (202-219-2148;
<Elizabeth.Albro@ed.gov>)

### A. Purpose

Through its Research on Reading and Writing (Read/Write) grants program,
the Institute intends to contribute to improvement of reading and
writing skills by (1) identifying curriculum and instructional practices
that are associated with better reading or writing outcomes as well as
mediators and moderators of the relations between these practices and
student outcomes; (2) developing new curricula or instructional
approaches for teaching individuals reading or writing skills or for
addressing the underlying causes of reading or writing difficulties
(e.g., poor oral language skills); (3) evaluating fully developed
curricula or instructional approaches for teaching reading or writing
skills, or for reducing/preventing reading or writing difficulties
through efficacy or replication trials; (4) evaluating the effectiveness
of curricula or instructional approaches for teaching reading or writing
skills that are implemented at scale; and (5) developing and validating
assessments that can be used in instructional settings to identify
sources of reading and writing difficulties.

The long-term outcome of this program will be an array of tools and
strategies (e.g., assessments, instructional approaches) that have been
documented to be effective for improving reading and writing.

### B. Background

Too many students are unable to understand what they read. According to
the 2005 National Assessment of Educational Progress (NAEP), 36 percent
of fourth graders, 27 percent of eighth graders, and 27 percent of
twelfth graders cannot read at the basic level. That is, when reading
grade appropriate text, these students cannot extract the general
meaning or make obvious connections between the text and their own
experiences, or make simple inferences from the text. In other words,
they cannot understand what they have read. By fourth grade, students
are expected to learn new information by reading subject matter
textbooks (Chall, 1996). Poor reading skills may hinder students\'
progress in learning academic content in all areas.

A similar picture emerges in the development of writing skills.
According to the 2002 NAEP writing assessment, 14 percent of fourth
graders, 15 percent of eighth graders, and 26 percent of twelfth graders
cannot write at the basic level.

Although tremendous advances have been made in understanding how
children learn to read, we have less systematic knowledge about how
individuals become proficient readers or proficient writers. There is
subsequently less agreement as to what a teacher can or should do to
cultivate active, engaged, and proficient readers and writers. On the
2005 NAEP, only 32 percent of fourth graders and 31 percent of eighth
graders were reading at the proficient or advanced levels. On the 2002
NAEP, 36 percent of twelfth graders were reading at the proficient or
advanced levels. With regard to writing, on the 2002 NAEP, 28 percent of
fourth graders, 31 percent of eighth graders, and 24 percent of twelfth
graders were at the proficient or advanced levels.

The Institute intends for the Reading and Writing program to support
research on the identification, development, and evaluation of
curricula, instructional approaches, and assessments designed to support
the development of proficient readers and writers from kindergarten
through postsecondary education.

The Institute encourages researchers to consider multivariate analyses
of district or state databases in order to identify existing programs
and practices that may be associated with better reading or writing
outcomes, and to examine factors and conditions that may mediate or
moderate the relations between the students outcomes and these programs
and practices. Another approach to the identification of potentially
effective instructional practices is for researchers to conduct
detailed, quantifiable observational measures of reading instruction
(types of instruction, frequency, duration, under what circumstances),
and then use the instructional data in conjunction with child
characteristics to predict subsequent reading performance (e.g., Connor,
et al., 2007). The goal here is to identify what type or combinations of
instructional activities are associated with better student outcomes and
for which students. Researchers following this strategy who can
successfully predict student performance could use this information as
the basis for developing an intervention.

In addition to the identification, development, and evaluation of
curricula and instructional approaches for improving reading and writing
skills, the Institute intends for the Reading and Writing program to
address the need to develop and validate reading and writing measurement
tools for classroom assessments to be used for instructional purposes
(e.g., progress monitoring). To improve reading and writing skills,
instruction may need to be tailored to the sources of difficulty that
individual students experience. An ideal learning environment might
involve regular and frequent assessment of skills and the possibility of
individualized instruction for students based on the particular source
of their difficulties. The Institute intends to support the development
of diagnostic assessments in reading and writing and assessments to
monitor progress in reading and writing. In addition, the Institute will
support the development, modification, and validation of assessments in
reading and writing for English learners.

### C. Specific Requirements

For the FY 2008 Reading and Writing topic, applicants must submit under
*either* [Goal One](#PIII_B_Goal1) *or* [Goal Two](#PIII_C_Goal2) *or*
[Goal
Three](#d.-requirements-for-goal-three-efficacy-and-replication-trials)
*or* [Goal Four](#PIII_E_Goal4) *or* [Goal
Five](#f.-requirements-for-goal-five-measurement-projects-for-topics-1-3-6-7-10-13).
More details on the requirements for each goal are listed in the section
on [General Requirements of the Proposed
Research](#general-requirements-of-the-proposed-research). In this
section, specific requirements that apply to applications to the Reading
and Writing topic are described.

Under the Read/Write program, applications must address:

-   reading or writing curricula designed to support the development of
    proficient readers or writers from kindergarten through high school;
    or

-   instructional approaches for teaching reading or writing that could
    be implemented within the context of an existing reading or writing
    program from kindergarten through high school;

-   at the postsecondary level, English composition courses intended to
    teach basic writing skills (e.g., instruction in grammar,
    organization, audience, style, and writing clear prose). Proposals
    to conduct research on curricula or instructional approaches for
    teaching creative writing or literature will not be considered; or

-   reading or writing assessments to support instruction from
    kindergarten through high school or to support basic writing
    instruction at the postsecondary level.

Researchers who are interested in proposing to develop or evaluate
curricula or instructional approaches targeting struggling adolescent or
adult readers or writers should apply to the Interventions for
Struggling Adolescent and Adult Readers and Writers program.

Researchers, who are interested in identifying underlying or component
processes of reading or writing, and the relations of these processes to
proficiency in reading or writing, should refer to the Cognition and
Student Learning research program.

# PART III. REQUIREMENTS OF THE PROPOSED RESEARCH

## **[14. GENERAL REQUIREMENTS OF THE PROPOSED RESEARCH]{.underline}**

# 

### A. Basic Requirements

**a. *Resubmissions***. Applicants who intend to revise and resubmit a
proposal that was submitted to one of the Institute's FY 2007
competitions but that was not funded must indicate on the application
form that their FY 2008 proposal is a revised proposal. Their FY 2007
reviews will be sent to this year\'s reviewers along with their
proposal. Applicants should indicate the revisions that were made to the
proposal on the basis of the prior reviews using no more than 3 pages of
Appendix A.

**b. *Applying to multiple topics****.* Applicants may submit proposals
to more than one of the Institute\'s FY 2008 competitions or topics. In
addition, within a particular competition or topic, applicants may
submit multiple proposals. However, applicants may submit a given
proposal only once (i.e., applicants may not submit the same proposal or
very similar proposals to multiple topics or to multiple goals in the
same topic or to multiple competitions). If the Institute determines
prior to panel review that an applicant has submitted the same proposal
or very similar proposals to multiple topics within or across
competitions and the proposal is judged to be compliant and responsive
to the submission rules and requirements described in the Request for
Applications, the Institute will select one version of the application
to be reviewed by the appropriate scientific review panel. If the
Institute determines after panel review that an applicant has submitted
the same proposal or very similar proposals to multiple topics within or
across competitions and if the proposal is determined to be worthy of
funding, the Institute will select the topic under which the proposal
will be funded.

**c. *Applying to a particular goal within a topic*.** To submit an
application to one of the Institute\'s research programs, applicants
must choose the specific goal under which they are applying. Each goal
has specific requirements.

**d. *Determining which goal is most appropriate for the proposed
project***. Applicants should read carefully the requirements for each
Goal and the examples of appropriate projects under each Goal. The
Institute strongly encourages potential applicants to contact the
relevant program officer listed in [Section
28](#inquiries-may-be-sent-to) if they have any questions regarding the
appropriateness of a particular project for submission under a specific
goal.

[]{#PIII_B_Goal1 .anchor}

### B. Requirements for Goal One (Identification Projects) 

*Because the requirements for Goal One are essentially the same across
the Institute\'s research grant topics, a generic description is used in
the funding announcement. Consequently, the examples provided may not
apply to a particular topic.*

[]{#PIII_B_Goal1a .anchor}**a. *Purpose of identification studies*.**
Through all of its research programs that include the Identification
goal (Goal One), the Institute is interested in the identification of
existing programs and practices that may be associated with better
academic outcomes and examination of factors and conditions that may
mediate or moderate the relations between student outcomes and these
programs and practices.

For Goal One, the Institute invites applications to conduct analyses of
multivariate data, such as longitudinal individual student data that
exist in a number of federal-, state-, and district-level databases.
Using existing longitudinal data sets, investigators are able to
capitalize on natural variation or discontinuities in education
practices. For example, in a particular year, a large district might
have implemented an intervention (e.g., curriculum, program, policy) at
the beginning of a specific year. An investigator might propose
interrupted time series analyses of the district\'s longitudinal
datasets to examine changes in student outcomes that follow the
implementation of the new intervention.

The strongest approaches to statistical modeling of multivariate data
involve testing two or more models of relationships using the same data.
Because multivariate analyses cannot fully adjust for selection biases
and the effects of variables that were not measured or were not measured
well, they are seldom if ever sufficient to support strong causal
conclusions about what works. However, when two or more models of
relationships among variables are tested with the same data, it may be
possible to determine that one is more plausible than another, thus
providing information relevant to understanding what does not work, as
well as what does work. That, in turn, can direct future efforts in
avenues that are more likely to be productive.

As an alternative to analyzing existing longitudinal databases,
applicants may propose to conduct a small scale descriptive longitudinal
study with primary data collection in which they attempt to predict
student outcomes based on differences in observed education practices.
For example, a researcher might propose to conduct detailed,
quantifiable observational measures of instructional practices (types of
instruction, frequency, duration, under what circumstances), and then
use the instructional data in conjunction with child characteristics to
predict subsequent student performance. The objective here is to
identify what type or combinations of instructional activities are
associated with better student outcomes and for which students.
Researchers following this strategy who can successfully predict student
performance could use this information as the basis for developing an
intervention (see, e.g., Connor, et al., 2007).

Evidence obtained through a Goal One project of the association between
exposure to a program or practice and better student outcomes has the
possibility of being used to support a subsequent application for a
[Goal Two](#PIII_C_Goal2) (Development) or [Goal
Three](#d.-requirements-for-goal-three-efficacy-and-replication-trials)
(Efficacy) project.

*By addressing the theoretical and empirical rationale for the study and
the practical importance of the intervention (e.g., program, practice)
that will be examined, Goal One applicants are addressing the
significance of their proposal.*

**b. *Methodological requirements***. **For all applications, including
those submitted under Goal One, the proposed research design must be
appropriate for answering the research questions or hypotheses that are
posed.**

\(i\) *Research questions*. Applicants should pose clear, concise
hypotheses or research questions.

\(ii\) *Database*. Applicants proposing secondary data analyses should
describe clearly the database(s) to be used in the investigation
including information on sample characteristics, variables to be used,
and ability to ensure access to the database if the applicant does not
already have access to it. The database should be described in
sufficient detail so that reviewers will be able to judge whether or not
the proposed analyses may be conducted with the database. If multiple
databases will be linked to conduct analyses, applicants should provide
sufficient detail for reviewers to be able to judge the feasibility of
the plan. If the applicant does not currently have access to the
databases needed for the study, the applicant should provide sufficient
documentation (e.g., letters of agreement) to assure reviewers that
access can be obtained and the project can be carried out in a timely
fashion.

The applicant should describe the primary outcome measures to be used,
including reliability and validity. In particular, applicants should
provide sufficient information on the construct validity of the proposed
measures. For example, if the applicant proposes to use a state database
from which the primary outcome measure will be performance on a reading
or mathematics achievement measure, the applicant should detail the
standardized measure from which the reading or mathematics scores are
derived.

\(iii\) *Primary data collection*. Applicants may propose a Goal One
project in which the primary focus is on the collection and analysis of
original data. The applicant should carefully describe the sample,
measures (including reliability and validity), and procedures proposed
for the primary data collection. Because Goal One projects must be
designed to predict student outcomes, if observational data are
collected, applicants should describe how the data would be collected
(e.g., procedures for maintaining inter-observer reliability), coded,
and quantified to allow quantitative analyses predicting the relation
between what was observed and student outcomes.

Applicants may also propose to collect original data as a supplement to
be used with an existing longitudinal database in order to answer the
question of interest. In such cases, applicants should describe the
sample and how the sample is related to or links to the proposed
secondary database, the measures to be used (including information on
the reliability and validity of the proposed instruments), and data
collection procedures.

\(iv\) *Data analysis*. The applicant must include detailed descriptions
of data analysis procedures. Because predictor variables relevant to
education outcomes (e.g., student, teacher, or district characteristics)
often covary, the Institute expects investigators to utilize the most
appropriate state-of-the-art analytic techniques to isolate the possible
effects of variables of interest. Analytic strategies should allow
investigators to examine mediators and moderators of programs and
practices. The relation between hypotheses, measures, independent and
dependent variables should be well specified. Strong applications will
include an explicit discussion of how exclusion from testing, or missing
data, will be handled within the statistical analyses. Strong
applications will propose an approach for comparing hypotheses or models
of relationships among variables.

**c. *Personnel and resources***. Competitive applicants will have
research teams that collectively demonstrate expertise in: (a) the
relevant student outcome (e.g., reading, mathematics, student
behaviors); (b) the type of intervention under investigation (e.g.,
curriculum, program, policy); (c) implementation of, and analysis of
results from, the research design that will be employed; and (d) working
with teachers, schools, or other education delivery settings that will
be employed if original data will be collected. Competitive applicants
will have access to institutional resources that adequately support
research.

**d. *Awards***. Typical awards for projects at this level are \$100,000
to \$350,000 (total cost = direct + indirect costs) per year. For
applicants proposing to do primarily secondary data analysis, the
maximum duration of the award is 2 years. Applicants proposing to do
short-term longitudinal studies may request up to 2 additional years
(i.e., the maximum duration of the award is 4 years) and additional
funds, but must justify the need for the additional time and funding.
The size of the award depends on the scope of the project.

### C. Requirements for Goal Two (Development Projects) 

*Because the requirements for Goal Two are essentially the same across
the Institute\'s research grant topics, a generic description is used in
the funding announcement. Consequently, the examples provided may not
apply to a particular topic.*

[]{#PIII_C_Goal2 .anchor}**a. *Purpose of Goal Two (Development).***
Through all of its research programs that include the Development goal
(Goal Two), the Institute intends to support the development of
education interventions -- curricula, instructional approaches and
programs. The Institute stresses that Goal Two applications are about
development, rather than demonstrations of the efficacy of an
intervention. Under Goal Two, the Institute does not intend to support
applications that propose to allocate substantial resources for testing
the effect of the proposed intervention. For example, the Institute does
not intend to support under Goal Two applications in which the
researcher proposes to spend one year developing the intervention and
the second and third years on testing the effect of the intervention in
a significant number of classrooms or schools. Applicants who have an
intervention that could be tested for efficacy should apply to Goal
Three.

From the Institute\'s standpoint, a funded development project would be
successful if at the end of a 1- to 3-year development award, the
investigators had a fully developed version of the proposed
intervention, including prototypes of all materials and products
necessary for implementation of the intervention in authentic education
delivery settings, and evidence demonstrating the feasibility of its
implementation in an authentic education delivery setting. The Institute
anticipates that investigators with successful development projects
would submit proposals to subsequent competitions for Goal Three
(Efficacy) awards.

[]{#PIII_C_Goal2b .anchor}**b. *Requirements for proposed
intervention***. Under Goal Two, the Institute invites applications to
develop new interventions or further develop interventions that are in
the early stages of development (e.g., those that do not have an entire
program or product ready to evaluate). It is important for applicants to
provide a strong rationale to support the development of the proposed
intervention. In essence, applicants are answering the question: *Why is
the proposed intervention likely to produce better student outcomes
relative to current education practices?*

In strong applications, researchers provide context for the proposed
intervention by including data on, or reviewing research describing, the
attributes of typical existing practices. Understanding the shortcomings
of current practice contributes to the rationale for the proposed
intervention.

Applicants should clearly describe the intervention and the logic model
for the intervention. For example, how do the features or components of
the intervention relate to each other temporally (or operationally),
pedagogically, and theoretically (e.g., why does A lead to B)?
Applicants should provide a strong theoretical and empirical
justification for the design and sequencing of the features or
components of the intervention. When applicants clearly describe the
logic model that guides the intervention and the specific features
making up the intervention, reviewers are better able to evaluate (a)
the relation between the theoretical and empirical foundation for the
intervention and the intervention (e.g., is the proposed intervention a
reasonable operationalization of the theory?) and (b) the relation
between the intervention and the outcome measures (e.g., do the proposed
measures tap the constructs that the intervention is intended to
address?).

Applicants should explain why the proposed intervention is likely to
produce substantially better student outcomes relative to current
practice. By clearly describing the intervention -- particularly, the
unique features of the intervention (\"active ingredients\") that are
hypothesized to produce the desired improvement in student outcomes --
as well as the typical existing practices, reviewers are better able to
judge whether the proposed intervention has the potential to produce
substantially better student outcomes because it is sufficiently
different from current practices and has \"active ingredients\" that
appear on the basis of theoretical or empirical reasons to be powerful
agents for improving student learning.

In the rationale to support the proposed intervention, applicants should
address the *practical* importance of the proposed intervention. For
example, when the proposed intervention is fully developed, will it have
the potential to improve students\' achievement scores in educationally
meaningful increments, if it were implemented over the course of a
semester or school year? In addition, would the proposed intervention be
both affordable for schools and easily implemented by schools (e.g., not
involve major adjustments to normal school schedules)?

*By describing (a) the intervention (e.g., features, components) and the
logic model for the intervention, (b) the theoretical and empirical
support for the proposed intervention, and (c) the practical importance
of the intervention,* *Goal Two applicants are addressing aspects of the
significance of their proposal.*

**c.** ***Methodological requirements***. **For all applications,
including those submitted under Goal Two, the proposed research design
must be appropriate for answering the research questions or hypotheses
that are posed.**

For Goal Two projects, applicants must clearly address the proposed
methods for developing the intervention and testing the feasibility of
implementation of the prototype in an authentic education delivery
setting. Applicants should describe the systematic process they will use
to collect empirical data that will provide feedback for refining the
intervention. A major objective of Goal Two projects is to refine and
improve the initial version of the intervention by implementing it, or
components of it, observing its functioning, and making necessary
adjustments in the design of the intervention so that it functions more
as intended.

Strong applications include clear descriptions of the development
activities so that reviewers will understand (a) what will be developed,
(b) how it will be developed, and (c) when the development will take
place. Applicants should describe what they would measure or observe to
determine whether the intervention is working as intended when they are
testing the feasibility of successive versions of the intervention. A
useful by-product of such testing is a set of fidelity of intervention
measures that could be used if the intervention were evaluated in an
efficacy trial (see [Goal
Three](#d.-requirements-for-goal-three-efficacy-and-replication-trials)).

A timeline that delineates the iterative process of drafting and
revising the intervention (e.g., features or components of the
intervention, procedures, training activities, and materials) is often a
simple way of showing reviewers how research activities will feed into
subsequent development (refinement) activities, so that information can
be used to make decisions and improvements. A variety of methodological
strategies may be employed during this phase. *For Development projects,
reviewers need to understand* *the iterative development process to be
used in the design and refinement of the proposed intervention.*

By the end of a Goal Two project, the Institute expects investigators to
have a fully developed intervention and demonstrated that the
intervention can be implemented in an authentic education delivery
setting.

\(i\) *Sample*. The applicant should define, as completely as possible,
the samples and settings that will be used to assess the feasibility and
usability of the intervention.

\(ii\) *Research plan*. The applicant must provide a detailed research
plan in which they detail the proposed procedures for developing the
intervention. Strong applications will include clear descriptions of:
(a) what needs to be developed; (b) the procedures for developing the
intervention; and (c) the procedures (including sample, measures, and
procedures for analyzing data) for determining if the intervention is
functioning as intended (e.g., Does the software program crash when
students use it? Are the activities planned for a particular lesson
do-able within the allotted time?). *Applicants should describe the
iterative development process to be used in the design and refinement of
the proposed intervention, and plans for acquiring evidence about the
operation of the intervention according to the logic model that they
describe.*

\(iii\) *Measures*. Goal Two projects typically rely on the collection
of process data that can help the researcher refine the intervention and
provide insight into the feasibility and usability of the proposed
intervention in authentic education delivery settings. Applicants should
clearly describe (a) what needs to be observed in order to determine if
the intervention is operating as intended and (b) how those observations
will be collected. Observational, survey, or qualitative methodologies
are encouraged to identify conditions that hinder implementation of the
intervention.

**d. *Personnel and resources***. Competitive applicants will have
research teams that collectively demonstrate expertise in: (a) the
relevant content area (e.g., reading, mathematics, student behaviors);
(b) type of intervention to be developed; (c) implementation of, and
analysis of results from, the research design that will be employed; and
(d) working with schools and other education delivery settings.
Competitive applicants will have access to institutional resources that
adequately support research.

An applicant may be or may involve *for-profit entities* in the project.
Involvement of the commercial developer or distributor must not
jeopardize the objectivity of the research. *Collaborations including
for-profit developers or distributors of education products must justify
the need for Federal assistance to undertake the evaluation of programs
that are marketed to consumers and consider cost-sharing part of the
cost of the evaluation.*

Applicants who previously or currently hold development grants with the
Institute should describe the results and outcomes of those grants to
date. They should indicate whether what was developed has been (or is
being) evaluated for efficacy ([Goal
Three](#d.-requirements-for-goal-three-efficacy-and-replication-trials))
and if results are available, what the results of those efficacy
evaluations have been. The Institute intends to support researchers
under Goal Two who can demonstrate their ability to develop
interventions that can be used in the field and tested for efficacy.

**e. *Awards***. Typical awards for projects at this level are \$150,000
to \$500,000 (total cost = direct + indirect costs) per year.
Development projects are for a maximum of 3 years. Development costs
vary according to the type of intervention that is proposed. Larger
awards will be considered. In all cases, the size of the award depends
on the scope of the project.

### D. Requirements for Goal Three (Efficacy and Replication Trials) 

*Because the requirements for Goal Three are essentially the same across
the Institute\'s research grant topics, a generic description is used in
the funding announcement. Consequently, the examples provided may not
apply to a particular topic.*

Under Goal Three, the Institute requests proposals to test the efficacy
of fully developed interventions. By *efficacy*, the Institute means the
degree to which an intervention has a net positive impact on the
outcomes of interest in relation to the program or practice to which it
is being compared.

**a.** ***Purpose of efficacy and replication trials***. Through all of
its research programs that include the Efficacy and Replication goal
(Goal Three), the Institute intends to fund efficacy trials to determine
whether or not fully-developed interventions -- programs, practices --
are effective under specified conditions (e.g., urban schools with a
high turnover rate among teachers), and with specific types of students
(e.g., English language learners). Results from efficacy projects have
less generalizability than results from effectiveness (scale-up)
evaluations under [Goal Four](#PIII_E_Goal4). The limited
generalizability can arise both from the lack of a full range of types
of settings and participants in the study, as well as through the
intensive involvement of the developers and researchers in the
implementation of the intervention. A well-designed efficacy trial
provides evidence on whether an intervention **can** work, but not
whether it would work if deployed widely. Under Goal Three, applicants
may propose an efficacy trial to determine if an intervention will work
under specific conditions or a replication trial to determine if an
intervention shown to produce a net positive impact in one setting will
produce a net positive impact under different conditions (e.g., with a
different population of students).

Under Goal Three, an applicant might propose to examine the efficacy of
the intervention in an experimental study in which, for example, half of
the classrooms are randomly assigned to the intervention program and
half are assigned to continue using standard district practices.
Alternatively, if the research team hypothesized that a variation in the
delivery of the program might improve the impact of the intervention,
the team might propose instead to randomly assign: (a) one-third of the
classrooms to the basic intervention; (b) one third of the classrooms to
the variation; and (c) one-third of the classrooms to continue with
standard district practices. *Applicants should use the efficacy and
replication trials to determine the conditions, if any, under which an
intervention produces meaningful improvement on academic outcomes.*

Also of interest to the Institute are proposals to compare the impact of
two interventions that are based on different theoretical models. In
such cases, the purpose might be to compare the efficacy of two
well-developed approaches to improving student learning. One advantage
to this approach is that, relative to designs in which the comparison
group experiences whatever the school or district currently provides
(but see the discussion of \"[business-as-usual](#Business_as_usual)\"
treatments below), the investigator should have better knowledge of the
critical components of each intervention and can attempt to create two
conditions in which, for example, instruction varies on a number of
critical components.

From the Institute\'s standpoint, a funded Efficacy/Replication project
would be *methodologically successful* if at the end of the grant
period, the investigators had rigorously evaluated the impact of a
clearly specified intervention on relevant student outcomes and under
clearly described conditions using a research design that meets the
Institute\'s What Works Clearinghouse standards
([http://whatworks.ed.gov](http://whatworks.ed.gov/)), whether or not
the intervention is found to improve student outcomes relative to the
comparison condition. The Institute would consider methodologically
successful projects to be *pragmatically successful* if the rigorous
evaluation determined that the intervention has a net positive impact on
student outcomes in relation to the program or practice to which it is
being compared.

[]{#PIII_D_Goal3b .anchor}**b. *Requirements for proposed
intervention***. Interventions appropriate for study under Goal Three
are interventions that are fully developed and have evidence of their
feasibility for use in authentic education delivery settings.

\(i\) Applicants must have an intervention that is fully developed and
ready to be evaluated. Applicants who intend to devote a significant
part of the project period to developing new components or materials for
the intervention or new delivery approaches should apply to [Goal
Two](#PIII_C_Goal2). Goal Three projects are limited to those
interventions that are fully developed.

Applicants must provide evidence that the intervention can be
implemented in authentic education delivery settings -- that is,
evidence of the feasibility and usability of the proposed intervention
in authentic education delivery settings. The interventions may already
be in wide use in education setting or may be newly (but fully)
developed interventions.

Also appropriate for Goal Three applications are proposals to
*replicate* the efficacy of an intervention in a different setting. For
instance, in a previous study, the applicant could have demonstrated the
efficacy of an intervention in a small random assignment trial in an
urban school district, and a reasonable next step would be to
*replicate* these findings in a rural school district.

\(ii\) Applicants must provide a compelling rationale that justifies the
Institute\'s investment in the evaluation of the proposed intervention.
As justification for the evaluation of an intervention, the Institute
will accept conceptual arguments of the importance of evaluating the
proposed intervention because of its relevance to public policy or
current education practice as would be judged by practitioners and
policymakers. For example, the proposed intervention may already be
widely used but have not been rigorously evaluated (e.g., a commercially
distributed program, a specific education policy). To support this
argument, applicants might include documentation of the widespread use
(e.g., across multiple states, or a single large state) of the program
to justify the proposed efficacy evaluation.

> Alternatively, applicants could provide a strong rationale justifying
> the investment in the evaluation of the proposed intervention based on
> (a) the theoretical foundation on which the intervention was
> developed; (b) research on related interventions or components of the
> proposed interventions; and/or (c) empirical evidence of the effect or
> potential effect of the proposed intervention based on smaller scale
> studies. In such cases, the applicant needs to address the question:
> Why is this intervention likely to produce better student outcomes
> relative to current practice? In addition, such applicants should
> address the *practical* importance of the proposed intervention. For
> example, is the intervention sufficiently comprehensive to improve
> student outcomes on end-of-year assessments? Is there evidence
> indicating that the proposed intervention is sufficiently different
> from current practices to potentially improve student outcomes
> relative to current practices?

\(iv\) Applicants should clearly describe a logic model for the proposed
intervention (e.g., describing the features or components of the
intervention and how they relate to each other and to the intended
outcomes both temporally (or operationally) and theoretically (e.g., why
A leads to B)). When applicants clearly describe the model that guides
the intervention and the intervention itself (e.g., specific features or
components of the intervention), reviewers are better able to evaluate
the relation between the theoretical and empirical foundation for the
intervention and the intervention (e.g., is the proposed intervention a
reasonable operationalization of the theory?). Reviewers are also better
able to evaluate the relation between the intervention and the outcome
measures (e.g., do the proposed measures tap the constructs that the
intervention is intended to address?).

[]{.mark}

> Some interventions are designed to affect the teaching and learning
> environment and indirectly affect student outcomes. In such cases, it
> is important for applicants to be clear in their logic model of the
> mediators that the intervention is designed to affect and through
> which student outcomes are intended to be improved.

[]{.mark}

> Strong applications will also include detailed descriptions of what
> the comparison group experiences. By clearly describing the
> intervention and the comparable treatment that the comparison group
> will receive, reviewers are better able to judge whether: (a) the
> intervention is sufficiently different from the comparison treatment
> so that one might reasonably expect a difference in student outcomes,
> and (b) fidelity measures and observations of the comparison group are
> sufficiently comprehensive and sensitive to identify and document
> critical differences between what the intervention and comparison
> groups receive.

*By describing (a) the intervention (e.g., features, components) and the
logic model for the intervention, (b) the theoretical and empirical
support for the proposed intervention, and (c) the practical importance
of the intervention, Goal Three applicants are addressing aspects of the
significance of their proposal.*

[]{#PIII_D_Goal3_c .anchor}**c. *Methodological requirements*.** **Under
Goal Three, the proposed research design must be appropriate for
answering the research questions or hypotheses that are posed.**

\(i\) *Research questions*. Applicants should pose clear, concise
hypotheses or research questions.

\(ii\) *Sample*. The applicant should define, as completely as possible,
the sample to be selected and sampling procedures to be employed for the
proposed study, including justification for exclusion and inclusion
criteria. Additionally, the applicant should describe strategies to
increase the likelihood that participants will remain in the study over
the course of the evaluation (i.e., reduce attrition).

\(iii\) *Research design*. The applicant must provide a detailed
research design. Applicants should describe how potential threats to
internal and external validity would be addressed. Studies using
randomized assignment to treatment and comparison conditions are
strongly preferred. When a randomized trial is used, the applicant
should clearly state the unit of randomization (e.g., students,
classroom, teacher, or school); choice of randomizing unit or units
should be grounded in a theoretical framework. Applicants should explain
the procedures for assignment of groups (e.g., schools) or participants
to treatment and comparison conditions.[^1]

*Only in circumstances in which a randomized trial is not possible* may
alternatives that substantially minimize selection bias or allow it to
be modeled be employed. Applicants proposing to use a design other than
a randomized design must make a compelling case that randomization is
not possible. Acceptable alternatives include appropriately structured
regression-discontinuity designs or other well-designed
quasi-experimental designs that come close to true experiments in
minimizing the effects of selection bias on estimates of effect size. A
well-designed quasi-experiment is one that reduces substantially the
potential influence of selection bias on membership in the intervention
or comparison group. This involves demonstrating equivalence between the
intervention and comparison groups at program entry on the variables
that are to be measured as program outcomes (e.g., student achievement
scores), or obtaining such equivalence through statistical procedures
such as propensity score balancing or regression. It also involves
demonstrating equivalence or removing statistically the effects of other
variables on which the groups may differ and that may affect intended
outcomes of the program being evaluated (e.g., demographic variables,
experience and level of training of teachers, motivation of students).
Finally, it involves a design for the initial selection of the
intervention and comparison groups that minimizes selection bias or
allows it to be modeled. For example, a very weak quasi-experimental
design that would *not* be acceptable as evidence of program efficacy
would populate the intervention condition with teachers who volunteered
for the program to be evaluated, and would select comparison teachers
who had the opportunity to volunteer but did not. In contrast, an
acceptable design would select teachers in one particular geographical
area of a city to be in the intervention, whereas teachers in another
geographical area, known to be demographically similar, would be
selected to be in the comparison condition. In the former case,
self-selection into the intervention is very likely to reflect
motivation and other factors that will affect outcomes of interest and
that will be impossible to equate across the two groups. In the latter
case, the geographical differences between the participants in the two
groups would ideally be unrelated to outcomes of interest, and in any
case, could be measured and controlled for statistically.

\(iv\) *Power*. Applicants should clearly address the power of the
evaluation design to detect a reasonably expected and minimally
important effect. When applicants justify what constitutes a reasonably
expected effect, applicants should indicate clearly (e.g., including the
statistical formula) how the effect size was calculated.

> Many evaluations of education interventions are designed so that
> clusters or groups of students, rather than individual students, are
> randomly assigned to treatment and comparison conditions. In such
> cases, the power of the design depends in part on the degree to which
> the observations of individuals within groups are correlated with each
> other on the outcomes of interest. For determining the sample size,
> applicants need to consider the number of clusters, the number of
> individuals within clusters, the potential adjustment from covariates,
> the desired effect, the intraclass correlation (i.e., the variance
> between clusters relative to the total variance between and within
> clusters), and the desired power of the design (note, other factors
> may also affect the determination of sample size, such as using
> one-tailed vs. two-tailed tests, repeated observations, attrition of
> participants, etc.).[^2] Strong applications will include empirical
> justification for the intraclass correlation and anticipated effect
> size used in the power analysis.

\(v\) *Measures*. Measures of student outcomes should include relevant
standardized measures of student achievement in addition to other
measures of student learning and achievement that are more closely
aligned with the proposed intervention (e.g., researcher-developed
measures). The applicant should provide information on the reliability,
validity, and appropriateness of proposed measures. In strong
applications, investigators will make clear that the skills or content
the intervention is designed to address are captured in the various
measures that are proposed.

> Some interventions are designed to change directly the teaching and
> learning environment and indirectly affect student outcomes. In such
> cases, applicants should provide measures of the primary mediators
> (i.e., proximal outcomes), as well as measures of student outcomes.

\(vi\) *Fidelity of implementation of the intervention*. The applicant
should specify how the implementation of the intervention would be
documented and measured. In strong applications, investigators will make
clear how the fidelity measures capture the critical features of the
intervention. Investigators should propose research designs that permit
the identification and assessment of factors impacting the fidelity of
implementation.

\(vii\) []{#Business_as_usual .anchor}*Comparison group, where
applicable*. Comparisons of interventions against other conditions are
only meaningful to the extent that one can tell what comparison group
receives or experiences. Applicants should compare intervention and
comparison groups on the implementation of critical features of the
intervention so that, for example, if there is no observed difference
between intervention and comparison student outcomes, they can determine
if key elements of the intervention were also provided in the comparison
condition (i.e., a lack of distinction between the intervention
treatment and the comparison treatment).

> In evaluations of education interventions, individuals in the
> comparison group typically receive some kind of treatment; rarely is
> the comparison group a \"no-treatment\" control. For some evaluations,
> the primary question is whether the treatment is more effective than a
> particular alternative treatment. In such instances, the comparison
> group receives a well-defined treatment that is usually an important
> comparison to the target intervention for theoretical or pragmatic
> reasons. In other cases, the primary question is whether the treatment
> is more effective than what is generally available and utilized in
> schools. In such cases, the comparison group might receive what is
> sometimes called \"business-as-usual.\" That is, the comparison group
> receives whatever the school or district is currently using or doing
> in a particular area. Business-as-usual generally refers to situations
> in which the standard or frequent practice across the nation is a
> relatively undefined education treatment. However, business-as-usual
> may also refer to situations in which a branded intervention (e.g., a
> published curriculum or program) is implemented with no more support
> from the developers of the program than would be available under
> normal conditions. In either case, *using a business-as-usual
> comparison group is acceptable.* When business-as-usual is one or
> another branded intervention, applicants should specify the treatment
> or treatments received in the comparison group. In all cases,
> applicants should account for the ways in which what happens in the
> comparison group are important to understanding the net impact of the
> experimental treatment. As noted in the preceding paragraph, in strong
> applications, investigators propose strategies and measures for
> comparing the intervention and comparison groups on key features of
> the intervention.
>
> The purpose here is to obtain information useful for *post hoc*
> explanations of why the experimental treatment does or does not
> improve student learning relative to the counterfactual.
>
> Finally, the applicant should describe strategies they intend to use
> to avoid contamination between treatment and comparison groups.
> Applicants do not necessarily need to randomize at the school level to
> avoid contamination between groups. Applicants should explain and
> justify their strategies for reducing contamination.

\(viii\) *Mediating and moderating variables*. Observational, survey, or
qualitative methodologies are encouraged as a complement to experimental
methodologies to assist in the identification of factors that may
explain the effectiveness or ineffectiveness of the intervention.
Mediating and moderating variables that are measured in the intervention
condition that are also likely to affect outcomes in the comparison
condition should be measured in the comparison condition (e.g., student
time-on-task, teacher experience/time in position).

> The evaluation should be designed to account for sources of variation
> in outcomes across settings (i.e., to account for what might otherwise
> be part of the error variance). Applicants should provide a
> theoretical rationale to justify the inclusion (or exclusion) of
> factors/variables in the design of the evaluation that have been found
> to affect the success of education programs (e.g., teacher experience,
> fidelity of implementation, characteristics of the student
> population). The research should demonstrate the conditions and
> critical variables that affect the success of a given intervention.
> The most scalable interventions are those that can produce the desired
> effects across a range of education contexts.

\(ix\) *Data analysis*. All proposals must include detailed descriptions
of data analysis procedures. For quantitative data, specific statistical
procedures should be described. The relation between hypotheses,
measures, independent and dependent variables should be clear. For
qualitative data, the specific methods used to index, summarize, and
interpret data should be delineated.

> Most evaluations of education interventions involve clustering of
> students in classes and schools and require the effects of such
> clustering to be accounted for in the analyses, even when individuals
> are randomly assigned to condition. Such circumstances generally
> require specialized multilevel statistical analyses using computer
> programs designed for such purposes. Strong applications will provide
> sufficient detail for reviewers to judge the appropriateness of the
> data analysis strategy. For random assignment studies, applicants need
> to be aware that typically the primary unit of analysis is the unit of
> random assignment.

**d. *Personnel and resources*.** Competitive applicants will have
research teams that collectively demonstrate expertise in: (a) the
relevant content area (e.g., reading, mathematics, student behaviors);
(b) the type of intervention being evaluated (e.g., curriculum, teacher
professional development, policy); (c) implementation of, and analysis
of results from, the research design that will be employed; and (d)
working with schools and other education delivery settings. Competitive
applicants will have access to institutional resources that adequately
support research.

For Goal Three projects, an applicant may be or may involve developers
or distributors (*including for-profit entities*) in the project, from
having them as full partners in its proposal to using off-the-shelf
training materials without involvement of the developer or distributor.
Involvement of the developer or distributor must not jeopardize the
objectivity of the evaluation. *Collaborations including for-profit
distributors of curriculum materials should justify the need for Federal
assistance to undertake the evaluation of programs that are marketed to
consumers and consider sharing the cost of the evaluation.*

Competitive applicants will have access to institutional resources that
adequately support research activities and access to schools in which to
conduct the research. Strong applications will document the availability
and cooperation of the schools or other education delivery settings that
will be required to carry out the research proposed in the application
via a letter of support from the education organization.

**e. *Awards***. Typical awards for projects at this level will be
\$250,000 to \$750,000 (total cost = direct + indirect costs) per year
for a maximum of 4 years. Larger budgets will be considered if a
compelling case can be made for such support. The size of the award
depends on the scope of the project.

[]{#PIII_E_Goal4 .anchor}

### E. Requirements for Goal Four (Scale-up Evaluations) 

*Because the requirements for Goal Four are essentially the same across
the Institute\'s research grant topics, a generic description is used in
the funding announcement. Consequently, the examples provided may not
apply to a particular topic.*

**a. *Purpose of scale-up evaluations***. Through all of its research
programs that include the Scale-up Evaluations goal (Goal Four), the
Institute intends to support effectiveness evaluations of
interventions - programs, practices - to determine whether or not fully
developed interventions are effective when they are implemented under
conditions that would be typical if a school district or other education
delivery setting were to implement them (i.e., without special support
from the developer or the research team) across a variety of conditions
(e.g., different student populations, different types of schools). The
key differences between Scale-up Evaluations (Goal Four) and Efficacy
Evaluations ([Goal
Three](#d.-requirements-for-goal-three-efficacy-and-replication-trials)),
as the Institute uses these terms, have to do with the delivery of the
intervention and the diversity of the sample. Scale-up Evaluations
require that intervention be implemented at a distance from the
researcher/developer of the intervention. That is, the researchers must
not be heavily involved in making the intervention work. The
intervention must be implemented in the school or other authentic
education setting, as it would be if the school, or entity, had
purchased and implemented the intervention on its own without any
involvement in a research study. Second, Scale-up Evaluations require
sufficient diversity in the sample of schools, classrooms, or students
to ensure appropriate generalizability. Scale-up Evaluations typically
require a larger sample than an Efficacy Evaluation. For Scale-up
Evaluations, the primary question of interest is, \"Does this
intervention produce a net positive increase in student learning and
achievement relative to the control group?\" As is true for Goal Three
studies, for Goal Four studies, depending on the research question of
interest, the control group may receive a well-defined alternative
treatment, or may receive whatever programs and practices are already
currently available and utilized by schools ([business-as-usual control
group](#Business_as_usual)). Finally, the Institute invests in Scale-up
Evaluations for interventions that have strong prior evidence of the
efficacy of the intervention.

[]{#PIII_E_Goal4b .anchor}**b. *Requirements for proposed
intervention***. To be considered for Goal Four awards, applicants must
provide a clear rationale for the *practical* importance of the
intervention. Applicants should address three questions related to
practical importance. (i) Is the intervention likely to produce
educationally meaningful effects on outcomes that are important to
educational achievement (e.g., grades, achievement test scores) and,
therefore, are of interest to parents, teachers, and education decision
makers? (ii) Is the intervention reasonably affordable to schools and
other education delivery entities? (iii) Is the intervention designed so
that it is feasible for schools and other education delivery entities to
implement the intervention? In addition, applicants should clearly
describe the components of the intervention. Interventions appropriate
for study under Goal Four are interventions that are fully developed and
have strong evidence of the efficacy of the program on a limited scale.

\(i\) *Educationally meaningful effects*. Applicants must provide
*strong* evidence of the efficacy of the program as implemented on a
small scale to justify the proposal to conduct a large-scale evaluation
of the effectiveness of the intervention. As an example of strong
evidence of efficacy, an applicant might describe the results of two or
more small scale, rigorously conducted evaluations using random
assignment to intervention and comparison conditions in which the
efficacy of the intervention is demonstrated with different populations
(e.g., urban and rural school districts). Alternatively, a single
efficacy evaluation might have involved schools from more than one
district and included a diverse population of teachers and students and
alone could constitute sufficient evidence of the efficacy of the
intervention. Importantly, the evidence of efficacy must be based on the
results of randomized field trials, or well-designed quasi-experimental
evaluations. Strong applications will include information on the size
and statistical significance of the effects that were obtained through
efficacy trials. Effect sizes and confidence limits should typically be
calculated based on a unit of analysis that is the same as the unit of
random assignment. For example, the results of an efficacy trial in
which classrooms were assigned to conditions should be analyzed based on
classroom means rather than results from individual students. Applicants
should indicate clearly (e.g., including the statistical formula) how
the effect size was calculated when they use effect sizes as part of the
rationale for justifying their intervention. Furthermore, information on
effect sizes is more useful to reviewers when sufficient context for
interpreting the effect sizes is provided.

\(ii\) *Feasible implementation*. The materials, training procedures,
organizational arrangements, and all other aspects of the intervention
must be developed to the point where the intervention is ready to be
implemented under real-world circumstances in a real-world way. Strong
applications will provide reviewers with sufficient information to
evaluate whether implementation of the intervention is feasible for
schools and other education entities under normal conditions (i.e.,
without any support from the researchers or developers of the
intervention that would not typically be available to entities wanting
to implement the intervention outside of a research study). For example,
applicants might include results from prior efficacy trials indicating
the degree of support provided for the implementation of the
intervention and the level of fidelity attained across classrooms or
schools.

\(iii\) *Description of the intervention*. All applicants should clearly
describe the intervention (e.g., features, components). When applicants
clearly describe the intervention, reviewers are better able to evaluate
the relation between the intervention and the outcome measures (e.g., do
the proposed measures tap the constructs that the intervention is
intended to address?). Strong applications will also include detailed
descriptions of what the comparison group experiences. By clearly
describing the components of the intervention and the comparable
treatment (e.g., training program) that the comparison group will
receive, reviewers are better able to judge whether (a) the intervention
is sufficiently different from the comparison treatment so that one
might reasonably expect a difference in student outcomes, and (b)
fidelity measures and observations of the comparison group are
sufficiently comprehensive and sensitive to identify and document
critical differences between the intervention and comparison conditions.

[]{#PIII_E_Goal4c .anchor}**c. *Implementation of the intervention***.
One goal of scale-up evaluations of interventions is to determine if
programs are effective when the developers of the program do not provide
any more support than would be available under normal conditions. That
is, the program should be implemented as it would be if the schools or
other entities that are delivering the program were to obtain the
program on their own and decide to use it apart from participation in
any research and evaluation study. A second goal is to determine if
programs implemented under these conditions are effective in a variety
of settings. Interventions that are effective at scale are those that
can produce the desired effects across a range of education contexts.
For Goal Four, the applicant should detail the conditions under which
the intervention will be implemented -- including explicitly detailing
what involvement the researcher/developer will have in the
implementation of the intervention and justifying this level of
involvement -- and provide procedures that will capture the conditions
and critical variables that affect the success of a given intervention.

*By addressing the implementation of the intervention and the
requirements for the intervention in section [14.E.b](#PIII_E_Goal4b),
Goal Four applicants are addressing the significance of their proposal.*

**d. *Methodological requirements***. **Under Goal Four, the proposed
research design must be appropriate for answering the research questions
or hypotheses that are posed.** For the methodological requirements for
Goal Four projects, please refer to the [methodological
requirements](#PIII_D_Goal3_c) listed under Goal Three.

In addition, to the methodological requirements listed under Goal Three,
for Goal Four projects, strong applications will include a
Cost-Feasibility analysis to assess the financial costs of program
implementation and assist schools in understanding whether
implementation of the program is practicable given their available
resources. Data should be collected on the monetary expenditures for the
resources that are required to implement the program. Financial costs
for personnel, facilities, equipment, materials, and other relevant
inputs should be included. Annual costs should be assessed to adequately
reflect expenditures across the lifespan of the program. The Institute
is *not* asking applicants to conduct an economic evaluation of the
program (e.g., cost-benefit, cost-utility, or cost-effectiveness
analyses), although applicants may propose such evaluation activities if
desired.[^3]

**e. *Personnel and resources***. Competitive applicants will have
research teams that collectively demonstrate expertise in: (a) the
relevant content area (e.g., reading, mathematics, student behaviors);
(b) the type of intervention proposed (e.g., program, practice, policy);
(c) implementation of, and analysis of results from, the research design
that will be employed; and (d) working with schools and other education
delivery settings.

Competitive applicants will have access to institutional resources that
adequately support research activities and access to schools in which to
conduct the research. Strong applications will document the availability
and cooperation of the schools or other education delivery settings that
will be required to carry out the research proposed in the application
via a letter of support from the education organization.

An applicant may involve developers or distributors (*including
for-profit entities*) of the intervention in the project, from having
the developers as full partners in its proposal to using off-the-shelf
teacher training materials without involvement of the developer or
publisher. However, involvement of the developer or distributor must not
jeopardize the objectivity of the evaluation. Strong applications will
carefully describe the role, if any, of the developer/distributor in the
intervention. Developers may not provide any training or support for the
implementation that is not normally available to users of the
intervention. Applicants should describe how objectivity in the
evaluation would be maintained. Strong applications will assign
responsibility for random assignment to condition and data analyses to
individuals who are *not* part of the organization that developed or
distributes the intervention.

*Collaborations including for-profit distributors of materials should
justify the need for Federal assistance to undertake the evaluation of
programs that are marketed to consumers and consider sharing the cost of
the evaluation.*

**f. *Awards***. The scope of Goal Four projects may vary. A smaller
project might involve several schools within a large urban school
district in which student populations vary in terms of SES, race, and
ethnicity. A larger project might involve large numbers of students in
several school districts in different geographical areas.

Awards for Goal Four projects may go up to a limit of \$6,000,000 (total
cost = direct + indirect costs) over a 5-year period. Typical awards are
less. Awards depend in part on the number of sites, cost of data
collection, and cost of implementation. The size of the award depends on
the scope of the project.

### F. Requirements for Goal Five (Measurement Projects for Topics 1-3, 6-7, 10-13) 

In Section 14.F, the Institute specifies the requirements for Goal Five
projects for the following topics: (Topic 1) Reading and Writing; (Topic
2) Mathematics and Science Education; (Topic 3) Cognition and Student
Learning; (Topic 6) Social and Behavioral Context for Academic Learning;
(Topic 7) Education Leadership; (Topic 10) High School Reform; (Topic
11) Interventions for Struggling Adolescent and Adult Readers and
Writers; (Topic 12) Postsecondary Education; and (Topic 13) Education
Technology.

**a. *Requirements of proposed assessments***. Applicants under Goal
Five should propose to develop assessments that can be used in education
delivery settings. Applications that would be appropriate for
consideration under Goal Five include, but are not limited to: (a)
proposals to develop new assessments; (b) proposals to modify or adapt
existing assessments; and (c) proposals to adapt assessments originally
designed and used for research purposes for broader use in instructional
settings.

Applicants should provide a compelling rationale to support the
development of the proposed assessment. Reviewers will consider (a) the
strength of the theoretical foundation for the proposed assessment, (b)
the existing empirical evidence supporting the proposed assessment, and
(c) whether the proposed assessment duplicates existing assessments.
Applicants should clearly describe (1) the construct(s) to be measured,
and (2) the dimensions or components of the construct(s) to be measured.
Applicants should clearly describe the components of the assessment
instrument and their relation to the target construct(s). When
applicants clearly describe the components of the assessment, reviewers
are better able to evaluate the relation between the theoretical and
empirical foundation for the assessment and the assessment itself (e.g.,
does the proposed assessment capture critical skills?), and whether the
proposed assessment will meet the needs for which it is intended.

In developing these assessments, researchers should keep in mind the
pragmatic constraints (e.g., number of students, limited class time,
time required to train teachers to use the assessments, costs) that
teachers and administrators will consider to determine whether the
instrument is a viable option for use in classrooms and other education
delivery settings. Applications should provide sufficient description of
the proposed assessment, and how it could be utilized within education
delivery settings for reviewers to judge the practicality of the
proposed assessment for instructional purposes.

*By describing the theoretical and empirical support for the proposed
assessment, the practical utility of the assessment, and the components
of the assessment, applicants are addressing aspects of the significance
of their proposal.*

**b. *Methodological requirements*.** There are two aspects of the
research methodology that applicants must clearly address: (a) the
proposed methods for developing the assessment, and (b) the proposed
research methods for obtaining evidence of the *validity and
reliability* of the instrument.

\(i\) *[Assessment development]{.underline}*. Applicants must detail the
proposed procedures for developing the assessment. Strong applications
will include descriptions of: (a) the procedures for determining the
constructs that will be \"tapped\" by the instrument; (b) the procedures
for selecting items to be used in the assessment, including assessing
difficulty of selected items, and obtaining representative responses to
items; and (c) the process for determining the administrative procedures
for conducting the assessment (e.g., mode of administration,
inclusion/exclusion of individual test takers, and whether make-ups or
alternative administrative conditions will be allowed). Applicants
should describe the process they will use to collect empirical data that
will provide feedback for refining specific components of the
assessment. *Applicants should describe the iterative development
process to be used in the design and refinement of the proposed
measurement tool.*

\(ii\) *[Assessment evaluation]{.underline}*. Applicants must clearly
describe the research plans for determining the validity and reliability
of the instrument. Applicants should describe the characteristics, size,
and analytic adequacy of samples to be used in each study, including
justification for exclusion and inclusion criteria. Applicants should
describe detailed planned analytic methods (e.g., statistical and/or
psychometric models), plans for treatment of missing responses, and
criteria for interpreting results.

**c. *Personnel and resources*.** Competitive applicants will have
research teams that collectively demonstrate expertise in: (a) the
target domain (e.g., reading, mathematics, behaviors); (b) assessment;
(c) implementation of, and analysis of results from, the research design
that will be employed; and (d) working with teachers, schools, or other
education delivery settings in which the proposed assessment might be
used. Competitive applicants will have access to institutional resources
that adequately support research activities and access to schools in
which to conduct the research.

**d. *Awards*.** Typical awards under Goal Five will be \$150,000 to
\$400,000 (total cost = direct + indirect costs) per year for up to 4
years. Larger budgets will be considered if a compelling case can be
made for such support. The size of award depends on the scope of the
project.

# PART IV. GENERAL SUBMISSION AND REVIEW INFORMATION

## **[15. APPLICATIONS AVAILABLE]{.underline}**

Application forms and instructions for the electronic submission of
applications will be available for the programs of research listed in
this RFA from the following web site:

[http://www.Grants.gov](http://www.Grants.gov/)

by the following dates:

July 26, 2007 Application Deadline Date April 23, 2007

November 1, 2007 Application Deadline Date April 30, 2007

The application form approved for use in the competitions specified in
this RFA is the government-wide SF424 Research and Related (R&R) Form
(OMB Number 4040-0001).

## **[16. MECHANISM OF SUPPORT]{.underline}**

The Institute intends to award grants pursuant to this request for
applications. The maximum length of the award period varies by topic and
within topic by goal. The maximum award length for each goal within a
specific topic is specified in the award section for that topic and goal
and ranges from two to five years. Please see details for each topic and
goal in the [Requirements of the Proposed
Research](#general-requirements-of-the-proposed-research) section of the
announcement.

## **[17. FUNDING AVAILABLE]{.underline}**

The size of the award depends on the scope of the project. Please see
specific details in the [Requirements of the Proposed
Research](#general-requirements-of-the-proposed-research) section of the
announcement. Although the plans of the Institute include the research
programs (topics) described in this announcement, awards pursuant to
this request for applications are contingent upon the availability of
funds and the receipt of a sufficient number of meritorious
applications. The number of projects funded under a specific topic and
goal depends upon the number of high quality applications submitted to
that topic and goal. The Institute does not have plans to award a
specific number of grants under each particular topic and goal.

## **[18. ELIGIBLE APPLICANTS]{.underline}** 

For the research grant topics, applicants that have the ability and
capacity to conduct scientifically valid research are eligible to apply.
Eligible applicants include, but are not limited to, non-profit and
for-profit organizations and public and private agencies and
institutions, such as colleges and universities.

## **[19. SPECIAL REQUIREMENTS]{.underline}**

Research supported through this program must be relevant to U.S.
schools.

Recipients of awards are expected to publish or otherwise make publicly
available the results of the work supported through this program. The
Institute asks IES-funded investigators to submit voluntarily to the
Educational Resources Information Center (ERIC) an electronic version of
the author\'s final manuscript upon acceptance for publication in a
peer-reviewed journal, resulting from research supported in whole or in
part, with direct costs from the Institute. The author\'s final
manuscript is defined as the final version accepted for journal
publication, and includes all modifications from the peer review
process.

Applicants should budget for one meeting each year in Washington, DC,
with other grantees and Institute staff. At least one project
representative should attend the two-day meeting.

The Institute anticipates that the majority of the research funded under
this announcement will be conducted in field settings. Hence, the
applicant is reminded to apply its negotiated off-campus indirect cost
rate, as directed by the terms of the applicant\'s negotiated agreement.

Research applicants may collaborate with, or be, for-profit entities
that develop, distribute, or otherwise market products or services that
can be used as interventions or components of interventions in the
proposed research activities. Involvement of the developer or
distributor must not jeopardize the objectivity of the evaluation.
Applications from, or collaborations including, such organizations
should justify the need for Federal assistance to undertake the
evaluation of programs that are marketed to consumers and consider
sharing the cost of the evaluation, as well as sharing all or a
substantial portion of the cost of the implementation of the product
being evaluated (e.g., sharing the cost of textbooks for students).

The Institute strongly advises applicants to establish a written
agreement among all key collaborators and their institutions (e.g.,
principal and co-principal investigators) regarding roles,
responsibilities, access to data, publication rights, and
decision-making procedures within 3 months of receipt of an award.

## **[20. LETTER OF INTENT]{.underline}** 

A letter indicating an applicant's intent to submit an application is
optional, but encouraged, for each application. The letter of intent
form must be submitted electronically by the date listed at the
beginning of this document, using the instructions provided at:
[https://ies.constellagroup.com]()[.]{.underline}

The letter of intent should include:

-   Descriptive title;

-   Topic and goal that the applicant will address;

-   Brief description of the proposed project;

-   Name, institutional affiliation, address, telephone number and
    e-mail address of the principal investigator(s);

-   Name and institutional affiliation of any key collaborators and
    contractors;

-   Duration of the proposed project;

-   Estimated budget request for each year; and

-   Total budget request.

The project description should be single-spaced and should not exceed
one page (about 3,500 characters). Although the letter of intent is
optional, is not binding, and does not enter into the review of a
subsequent application, the information that it contains allows
Institute staff to estimate the potential workload to plan the review.

## **[21. SUBMITTING AN APPLICATION]{.underline}**

Applications must be submitted **electronically by 4:30
p.m**.,**Washington, DC time** by the application deadline date, using
the standard forms and the instructions provided at the following web
site:

[http://www.Grants.gov](http://www.Grants.gov/)

Potential applicants should check this site for information about the
electronic submission procedures that must be followed and the software
that will be required.

## **[22. CONTENTS OF APPLICATION]{.underline}** 

All applications and proposals for Institute funding must be contained
within specified page limits. Internet Web site addresses (URLs) may not
be used to provide information necessary to the review because reviewers
are under no obligation to view the Internet sites.

All of the instructions and requirements regarding (a) submission of the
application, (b) application page limits, (c) acceptable format, and (d)
necessary attachments (.PDF files) will be provided in the **Application
Instructions** document for this competition to be found under the "For
Applicants \-- Apply for Grants" link of Grants.gov. Also, all of the
required forms will be provided in the **Application Package** that
accompanies the Application Instructions.

You must search for the downloadable Application Instructions and
Application Package for each competition by the CFDA number. Do not
include the alpha suffix in your search (e.g., search for 84.305, not
84.305A). For this competition, make sure that you download the
"Education Research" Application Instructions and Application Package.

In this section, the Institute provides instructions regarding the
content of the (a) project summary/abstract, (b) project narrative, (c)
bibliography and references cited, (d) biographical sketches of
senior/key personnel, (e) narrative budget justification (f) subaward
budgets, (g) Appendix A, (h) Appendix B, (i) human subjects narrative,
and (j) additional forms. The instructions below will be reiterated in
the Application Instructions document for this competition, which will
be available, as noted above, under the "For Applicants \-- Apply for
Grants" link of Grants.gov.

### A. Project Summary/Abstract

The project summary/abstract will be submitted as a .PDF attachment, is
limited to 1 single-spaced page and must adhere to the margin, format,
and font size requirements described in the project narrative section.

The project summary/abstract should include (1) the title of the
project; (2) the RFA topic and goal under which the applicant is
applying (e.g., development, efficacy); and brief descriptions of (3)
the purpose (e.g., to develop and document the feasibility of an
intervention); (4) the setting in which the research will be conducted
(e.g., rural school districts in Alabama); (5) the population(s) from
which the participants of the study(ies) will be sampled (age groups,
race/ethnicity, SES); (6) if applicable, the intervention or assessment
to be developed or evaluated or validated; (7) if applicable, the
control or comparison condition (e.g., what will participants in the
control condition experience); (8) the primary research method; (9) if
applicable, measures of key outcomes; and (10) if applicable, data
analytic strategy.

### B. Project Narrative

The project narrative will be submitted as a .PDF attachment.
Incorporating the requirements outlined under the section on
[Requirements of the Proposed
Research](#general-requirements-of-the-proposed-research) and the
requirements listed under the Specific Requirements section of the
relevant research grant topic, the *project narrative* provides the
majority of the information on which reviewers will evaluate the
proposal.

The project narrative must include the four sections described below (a.
\"Significance\" through d. \"Resources\") in the order listed and must
conform to the format requirements described on the application
submission website.

The project narrative is limited to **25 single-spaced pages** for all
applicants. This 25-page limit does not include any of the SF 424 forms,
the one-page summary/abstract, the appendices, research on human
subjects information, bibliography and references cited, biographical
sketches of senior/key personnel, narrative budget justification, sub
award budget information or certifications and assurances. **Reviewers
are able to conduct the highest quality review when applications are
concise and easy to read, with pages numbered consecutively**.

For the purposes of applications submitted under this RFA, a "page" is
8.5 in. x 11 in., on one side only, with 1 inch margins at the top,
bottom, and both sides. Text must be single spaced in the narrative. To
ensure that the text is easy for reviewers to read and that all
applicants have the same amount of available space in which to describe
their projects, applicants must adhere to the type size and format
specifications for the entire narrative including footnotes**. It is
very important that applicants review carefully the "Application Format
Requirements" outlined in** ***Fiscal Year 2008 Application Package
Highlights***, which will be part of the application instructions,
available on [http://www.Grants.gov](http://www.Grants.gov/) by April
23, 2007.

**a. *Significance****.* In the [General Requirements of the Proposed
Research](#general-requirements-of-the-proposed-research) section and in
the subsections describing the requirements for the proposed
intervention for [Goal One](#PIII_B_Goal1), [Goal Two](#PIII_C_Goal2),
[Goal
Three](#d.-requirements-for-goal-three-efficacy-and-replication-trials),
[Goal Four](#PIII_E_Goal4), and [Goal
Five](#f.-requirements-for-goal-five-measurement-projects-for-topics-1-3-6-7-10-13),
the Institute details the information that the applicant should include
in order to address the significance of the proposed project.

For projects in which an intervention or assessment is proposed (whether
to be developed or to be evaluated), applicants may use Appendix B to
include up to 10 pages of examples of materials to be used by
participants (e.g., training materials for teachers, computer screens
depicting how information is presented to students, examples of test
items for a proposed assessment). Applicants should be aware that all
narrative text describing the theoretical background, empirical support,
components of the assessment or intervention, or any other aspect of the
proposal must be included within the 25-page project narrative. The only
materials that are allowed in Appendix B are examples of the materials
that are used by or presented to participants in the intervention or
assessment.

**b. *Methods.*** The Methods section of applications for [Goal
One](#PIII_B_Goal1), [Goal Two](#PIII_C_Goal2), [Goal
Three](#d.-requirements-for-goal-three-efficacy-and-replication-trials),
[Goal Four](#PIII_E_Goal4), and [Goal
Five](#f.-requirements-for-goal-five-measurement-projects-for-topics-1-3-6-7-10-13)
should address all of the requirements detailed in the methodological
requirements sections for the appropriate research goal.

**c. *Personnel****.* Applicants must include brief descriptions of the
qualifications of key personnel (information on personnel should also be
provided in their curriculum vitae) in the research narrative to be
compliant with the requirements of the Request for Applications. For
each of the key personnel, please describe the roles, responsibilities,
and percent of time devoted to the project.

**d. *Resources*.** Applicants must include a brief description of
resources available to support the project at the applicant's
institution and in the field settings in which the research will be
conducted in the research narrative to be compliant with the
requirements of the Request for Applications.

### C. Bibliography and References Cited 

This section will be submitted as a .PDF attachment. Please include
complete citations, including titles and all authors, for literature
cited in the research narrative.

### D. Biographical Sketches of Senior/Key Personnel 

This section will be submitted as a .PDF attachment. Abbreviated
curriculum vitae should be provided for the principal investigator(s)
and other key personnel. *Each vita is limited to 4 pages and should
include information sufficient to demonstrate that personnel possess
training and expertise commensurate with their duties (e.g.,
publications, grants, relevant research experience), and have adequate
time devoted to the project to carry out their duties. The fifth page of
the attachment should list current and pending grants with the
proportion of the individual\'s time allocated to each project.* The
curriculum vita must adhere to the margin, format, and font size
requirements described in the project narrative section.

### E. Narrative Budget Justification

This section will be submitted as a .PDF attachment and should provide
sufficient detail to allow reviewers to judge whether reasonable costs
have been attributed to the project. The budget justification should
correspond to the itemized breakdown of project costs that is provided
in the Research & Related Budget (SF 424) Sections A & B; C, D, & E; and
F-K. It should include the time commitments and brief descriptions of
the responsibilities of key personnel. For consultants, the narrative
should include the number of days of anticipated consultation, the
expected rate of compensation, travel, per diem, and other related
costs. A justification for equipment purchase, supplies, travel and
other related project costs should also be provided in the budget
narrative for each project year outlined in the Research & Related
Budget (SF 424).

For those applications that include a subaward(s) for work conducted at
collaborating institutions, the narrative should also provide the
details about the subaward(s). Include the actual subaward budgets as a
separate attachment. (See below [*"Subaward
Budget"*](#f.-subaward-budget).)

Applicants should use their institution's federal indirect cost rate and
use the off-campus indirect cost rate where appropriate (see
instructions under [Section 19 Special
Requirements](#special-requirements)). If less than 75 percent of total
indirect costs are based on application of the off-campus rate, the
applicant should provide a detailed justification.

### F. Subaward Budget 

This section will be submitted as a .PDF attachment. For applications
that include a subaward(s) for work conducted at collaborating
institutions, applicants must submit an itemized budget spreadsheet for
each subaward for each project year. As noted above, the details of the
subaward costs should be included in the [Narrative Budget
Justification](#e.-narrative-budget-justification). An Excel spreadsheet
will be provided in the electronic application package to allow
applicants to enter the subaward budget information in accordance with
the prescribed format. Applicants will complete the spreadsheet in Excel
format, convert it to a .PDF file, and then upload it as an attachment.

### G. Appendix A

Appendix A should be included at the end of the [Project
Narrative](#b.-project-narrative), and will be submitted as part of the
same .PDF attachment.

The purpose of A*ppendix A* is to allow the applicant to include any
figures, charts, or tables that supplement the research text, examples
of measures to be used in the project, and letters of agreement from
partners (e.g., schools) and consultants. In addition, in the case of a
resubmission, the applicant may use **up to 3 pages** of the appendix to
describe the ways in which the revised proposal is responsive to prior
reviewer feedback. These are the only materials that may be included in
Appendix A; all other materials will be removed prior to review of the
application. Narrative text related to any aspect of the project (e.g.,
descriptions of the proposed sample, the design of the study, or
previous research conducted by the applicant) must be included in the
research narrative. Letters of agreement should include enough
information to make it clear that the author of the letter understands
the nature of the commitment of time, space, and resources to the
research project that will be required if the application is funded. The
appendix is limited to 15 pages. The Institute recognizes that some
applicants may have more letters of agreement than will be accommodated
by the 15-page limit. In such instances, applicants should include the
most important letters of agreement and may list the letters of
agreement that are not included in the application due to page
limitations.

### H. Appendix B (optional)

If applicable, Appendix B should be included at the end of the [Project
Narrative](#b.-project-narrative), following [Appendix
A](#g.-appendix-a), and will be submitted as part of the same .PDF
attachment.

The purpose of Appendix B is to allow applicants who are proposing an
intervention or assessment to include examples of curriculum material,
computer screens, test items, or other materials used in the
intervention or assessment. These are the only materials that may be
included in Appendix B; all other materials will be removed prior to
review of the application. Appendix B is limited to 10 pages. Narrative
text related to the intervention (e.g., descriptions of research that
supports the use of the intervention/assessment, the theoretical
rationale for the intervention/assessment, or details regarding the
implementation or use of the intervention/assessment) must be included
in the 25-page research narrative.

### I. Research on Human Subjects

This section will be submitted as a .PDF attachment. If an applicant
proposes research activities involving human subjects at any time during
the proposed project period, either at the applicant organization or at
any other performance site or collaborating institution, then the
applicant must provide either a human subjects \"exempt research
narrative\" or a \"nonexempt research narrative" and upload this
narrative as instructed in the ***Fiscal Year 2008 Application Package
Highlights***. See the U.S. Department of Education's web page for
detailed information about the protection of human subjects in research:

<http://www.ed.gov/policy/fund/guid/humansub/overview.html>

### J. Additional Forms

Please note that applicants selected for funding will be required to
submit the following certifications and assurances before a grant is
issued:

> \(1\) SF 424B-Assurances-Non-Construction Programs
>
> \(2\) Grants.gov Lobbying Form
>
> \(3\) ED 80-0014 (if applicable)-Lower Tier Certification

(4) SF-LLL (if applicable) - Disclosure of Lobbying Activities

(5) Protection of Human Research Subjects assurance and/or Institutional
    Review Board certification, as appropriate

## **[23. APPLICATION PROCESSING]{.underline}** 

Applications must be received by **4:30 p.m. Washington, DC time** on
the application deadline date listed in the heading of this request for
applications. Upon receipt, each application will be reviewed for
compliance and for responsiveness to this request for applications.
Applications that do not address specific requirements of this request
will be returned to the applicants without further consideration.

## **[24. PEER REVIEW PROCESS]{.underline}**

Applications that are compliant and responsive to this request will be
evaluated for scientific and technical merit. Reviews will be conducted
in accordance with the review criteria stated below by a panel of
scientists who have substantive and methodological expertise appropriate
to the program of research and request for applications.

Each application will be assigned to one of the Institute\'s scientific
review panels. At least two primary reviewers will complete written
evaluations of the application, identifying strengths and weaknesses
related to each of the review criteria. Primary reviewers will
independently assign a score for each criterion, as well as an overall
score, for each application they review. Based on the overall scores
assigned by primary reviewers, an average overall score for each
application will be calculated and a preliminary rank order of
applications prepared before the full peer review panel convenes to
complete the review of applications.

The full panel will consider and score only those applications deemed to
be the most competitive and to have the highest merit, as reflected by
the preliminary rank order. A panel member may nominate for
consideration by the full panel any proposal that he or she believes
merits full panel review but would not have been included in the full
panel meeting based on its preliminary rank order.

## **[25. REVIEW CRITERIA FOR SCIENTIFIC MERIT]{.underline}**

The purpose of Institute-supported research is to contribute to the
solution of education problems and to provide reliable information about
the education practices that support learning and improve academic
achievement and access to education for all students. Reviewers for all
applications will be expected to assess the following aspects of an
application in order to judge the likelihood that the proposed research
will have a substantial impact on the pursuit of that goal. Information
pertinent to each of these criteria is also described above in the
section on [Requirements of the Proposed
Research](#general-requirements-of-the-proposed-research), in the
Specific Requirements section of the relevant research grant topic, and
in the description of the [project narrative](#b.-project-narrative),
which appears in the section on Contents of Application.

### A. Significance 

For significance of the project, Goal One applicants need to address the
theoretical and empirical rationale for the study and the practical
importance of the intervention (e.g., program, practice) that will be
examined issues, as outlined in section [III.14.B.a](#PIII_B_Goal1a)
(Purpose of identification studies).

[]{.mark}

For significance of the project, Goal Two and Goal Three applicants need
to describe (a) the intervention (e.g., features, components) and the
logic model for the intervention, (b) the theoretical and empirical
support for the proposed intervention, and (c) the practical importance
of the intervention, as detailed in section [III.14.C.b](#PIII_C_Goal2b)
(for Goal Two: Requirements for proposed intervention.) and in section
[III.14.D.b](#PIII_D_Goal3b) for Goal Three.

[]{.mark}

For significance of the project, Goal Four applicants need to address
the implementation of the intervention as discussed in section
[III.14.E.c](#PIII_E_Goal4c) and the requirements for the intervention
in section [III.14.E.b](#PIII_E_Goal4b).

[]{.mark}

For significance of the project, Goal Five applicants need to describe
the theoretical and empirical support for the proposed assessment, the
practical utility of the assessment, and the components of the
assessment.

[]{.mark}

### B. Research Plan 

Does the applicant address the requirements described in the
methodological requirements section for the Goal under which the
applicant is submitting the proposal?

### C. Personnel 

Does the description of the personnel make it apparent that the
principal investigator, project director, and other key personnel
possess the training and experience and will commit sufficient time to
competently implement the proposed research?

### D. Resources

Does the applicant have the facilities, equipment, supplies, and other
resources required to support the proposed activities? Do the
commitments of each partner show support for the implementation and
success of the project?

## **[26. RECEIPT AND START DATE SCHEDULE]{.underline}**

**A. Letter of Intent Receipt Dates:**

July 26, 2007 Application Deadline Date May 24, 2007

November 1, 2007 Application Deadline Date September 6, 2007

**B. Application Deadlines:**

Summer Deadline Date July 26, 2007

Fall Deadline Date November 1, 2007

**C. Earliest Anticipated Start Date:**

July 26, 2007 Application Deadline Date March, 2008

November 1, 2007 Application Deadline Date July, 2008

## **[27. AWARD DECISIONS]{.underline}**

The following will be considered in making award decisions:

-   Scientific merit as determined by peer review

-   Responsiveness to the requirements of this request

-   Performance and use of funds under a previous Federal award

-   Contribution to the overall program of research described in this
    request

-   Availability of funds

## **[28. INQUIRIES MAY BE SENT TO]{.underline}** 

For the convenience of applicants, in this section we provide contact
information for all of the NCER

##### research programs.

**A. Reading and Writing**

#### Dr. Elizabeth Albro

> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Elizabeth.Albro@ed.gov>
>
> Telephone: (202) 219-2148

**B. Mathematics and Science Education**

Dr. Christina Chhin

> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Christina.Chhin@ed.gov>
>
> Telephone: (202) 219-2280

### C. Cognition and Student Learning

#####  Dr. Carol O'Donnell 

> Institute of Education Sciences

#### 555 New Jersey Avenue, NW

> Washington, DC 20208
>
> Email:
> [[Carol.O'Donnell@ed.gov]{.underline}](mailto:Elizabeth.Albro@ed.gov)

Telephone: (202) 208-3749

**D. Teacher Quality (Reading and Writing and Mathematics and Science
Education)**

> Dr. Harold Himmelfarb
>
> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Harold.Himmelfarb@ed.gov>
>
> Telephone: (202) 219-2031

### E. Social and Behavioral Context for Academic Learning

> Dr. Christina Chhin
>
> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Christina.Chhin@ed.gov>
>
> Telephone: (202) 219-2280

**F. Education Leadership**

> Dr. Katina Stapleton
>
> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Katina.Stapleton@ed.gov>
>
> Telephone: (202) 219-2154

**G. Education Policy, Finance, and Systems**

> Dr. Katina Stapleton
>
> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Katina.Stapleton@ed.gov>
>
> Telephone: (202) 219-2154

**H. Early Childhood Programs and Policies**

> Dr. Caroline Ebanks
>
> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Caroline.Ebanks@ed.gov>
>
> Telephone: (202) 219-1401

**I. High School Reform**

> Dr. David Sweet
>
> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <David.Sweet@ed.gov>
>
> Telephone: (202) 219-1748

**J. Interventions for Struggling Adolescent and Adult Readers**

> Dr. Elizabeth Albro
>
> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Elizabeth.Albro@ed.gov>
>
> Telephone: (202) 219-2148

**K. Postsecondary Education**

> Dr. Ram Singh
>
> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Ram.Singh@ed.gov>
>
> Telephone: (202) 219-2025

**L. Education Technology**

> Dr. Edward Metz
>
> Institute of Education Sciences
>
> 555 New Jersey Avenue, NW
>
> Washington, DC 20208
>
> Email: <Edward.Metz@ed.gov>
>
> Telephone: (202) 208-1983

## **[29. PROGRAM AUTHORITY]{.underline}**

20 U.S.C. 9501 [et]{.underline} [seq]{.underline}., the "Education
Sciences Reform Act of 2002," Title I of Public Law 107-279, November 5,
2002. This program is not subject to the intergovernmental review
requirements of Executive Order 12372.

## **[30. APPLICABLE REGULATIONS]{.underline}** 

The Education Department General Administrative Regulations (EDGAR) in
34 CFR parts 74, 77, 80, 81, 82, 84, 85, 86 (part 86 applies only to
institutions of higher education), 97, 98, and 99. In addition 34 CFR
part 75 is applicable, except for the provisions in 34 CFR 75.100,
75.101(b), 75.102, 75.103, 75.105, 75.109(a), 75.200, 75.201, 75.209,
75.210, 75.211, 75.217, 75.219, 75.220, 75.221, 75.222, and 75.230.

## **[31. REFERENCES]{.underline}**

American Psychological Association, Research Office (2004). *2001
Doctorate* *Employment Survey*. Report by Jessica Kohout and Marlene
Wicherski. Washington, DC: American Psychological Association.

Anderson, J.R., Reder, L.M., & Simon, H.A. (2000, Summer). Applications
and Misapplications of Cognitive Psychology to Mathematics Education.
*Texas Educational Review*. Downloaded from
<http://act-r.psy.cmu.edu/publications/pubinfo.php?id=146> on March 6,
2006.

Barton, P. E. (2000). What jobs require: Literacy, education, and
training, 1940-2006. Washington, DC: Educational Testing Service.

Bennett, K. J., Brown, K. S., Boyle, M., Racine, Y., & Offord, D.
(2003). Does low reading achievement at school entry cause conduct
problems? *Social Science and Medicine, 56*, 2443-2448

Berne, R., & Stiefel, & L. (1997). Student-level school resource
measures. *Selected papers in school finance* (NCES 97-536).
<http://nces.ed.gov/pubs97/97536.pdf>.

Boruch, R., de Moya, D., and Snyder, B. (2001). The importance of
randomized field trials in

education and related fields. In R. Boruch and F. Mosteller (Eds.).
*Evidence Matters*.

Washington, DC: Brookings Institution Press.

Carlson, E. A., Sroufe, L. A., Collins, W. A., Jimerson, S., Weinfield,
N., Hennighausen, K., Egeland, B., Hyson, D. M., Anderson, F., & Meyer,
S. E. (1999). Early environmental support and elementary school
adjustment as predictors of school adjustment in middle adolescence.
*Journal of Adolescent Research, 14*, 72-94.

Carver, S. M., & Klahr, D. (Eds.). (2001). *Cognition and instruction:
Twenty-five years of progress*. Mahwah, NJ: Lawrence Erlbaum Associates,
Publishers.

Cepeda, N.J., Pashler, H., Wixted, J.T., & Rohrer, D. (in press).
Distributed practice: A review and quantitative synthesis.
*Psychological Bulletin*.

Chall, J.S. (1996). Stages of reading development. 2^nd^ edition. Fort
Worth, TX: Harcourt Brace.

Coalition for Evidence-Based Policy. (2002, November). Rigorous
evidence: The key to

progress in education? Lessons from medicine, welfare and other fields.
*Proceedings of*

*The Coalition for Evidence-Based Policy Forum*, Washington, DC.

College Board (2005a). *Trends in College Pricing 2005*. New York, NY:
College Board

Connor, C. McD., Morrison, F. J., Fishman, B. J., Schatschneider, C., &
Underwood, P. (January 26, 2007). *Science*, *315*, 464-465.

Cook, T.D. (2001). Sciencephobia: Why education researchers reject
randomized experiments.

*Education Next*. Retrieved January 5 from
[www.educationnext.org](http://www.educationnext.org/).

Cornwell, C. M., Mustard, D. B., & Sridhar, D. (2005, August). The
enrollment effects of merit-based financial aid: Evidence from Georgia's
Hope scholarship (available at the University of Georgia Web site:
[*http://www.terry.uga.edu/hope/hope.enrollments.pdf*](http://www.terry.uga.edu/hope/hope.enrollments.pdf),
accessed on September 5, 2005).

Donner, A., & Klar, N. (2000). *Design and Analysis of Cluster
Randomization Trials in Health Research*. New York: Oxford University
Press.

Ehrenberg, R. & Brewer, D. (1994) Do school and teacher characteristics
matter? Evidence from High School and Beyond*. Economics of Education
Review, 14*, 1-23.

Evertson, C.M. & Weinstein, C.S. (2006). Classroom management as a field
of inquiry. In C.M. Evertson & C.S. Weinstein. (Eds.), *Handbook of
Classroom Management: Research, Practice, and Contemporary Issues*.
Mahwah, NJ: Lawrence Erlbaum Associates

Gitomer, D. H., Latham, A. S., & Ziomek, R. (1999). *The Academic
Quality of Prospective Teachers: The Impact of Admissions and Licensure
Testing.* [www.ets.org/praxis/](http://www.ets.org/praxis/)

Goldhaber, D. D. & Brewer, D. J. (1997). Why don\'t schools and teachers
seem to matter? Assessing the impact of unobservables on education.
*Journal of Human Resources, 32*, 505-523.

Goldhaber, D. D. & Brewer, D. J. (2000). Does teacher certification
matter? High school certification status and student achievement.
*Educational Evaluation and Policy Analysis, 22* (2), 129-145.

Gottfredson, G. D., & Gottfredson, D. C. (2001). What schools do to
prevent problem behavior and promote safe environments. *Journal of
Educational and Psychological Consultation, 12*, 313-344.

Greenwald, R., Hedges, L., & Laine, R. (1996). The effects of school
resources on student achievement. *Review of Educational Research, 66*,
361-396.

Hess, F.M. and Kelly, A. P. (2005, May*) Learning to Lead? What Gets
Taught in Principal Preparation Programs* (PEPG 05-02). Cambridge, MA:
Program on Education Policy and Governance at Harvard University.

Hoffer, T.B., S. Sederstrom, L. Selfa, V.Welch, M. Hess, S. Brown, S.
Reyes, K. Webber, and

I. Guzman-Barron. (2003). *Doctorate Recipients from United States
Universities: Summary*

*Report 2002*. Chicago: National Opinion Research Center. (The report
provides the results of data collected in the Survey of Earned
Doctorates, conducted for six Federal agencies, NSF, NIH, USED, NEH,
USDA, and NASA by NORC.).

Horn, L., & Berger, R. (2004). *College Persistence on the Rise? Changes
in 5-Year Degree Completion and Postsecondary Persistence Rates between
1994 and 2000* (NCES 2005-156). U.S. Department of Education, National
Center for Education Statistics. Washington, DC: U.S. Government
Printing Office.

Horn, L. J., Chen, X., & Chapman, C. (2003). *Getting Ready to Pay for
College: What Students and Their Parents Know About the Cost of College
Tuition and What They Are Doing to Find Out,* NCES 2003--030. U.S.
Department of Education, National Center for Education Statistics.
(2003). Washington, DC: 2003.

Kemple, J.J., Herlihy, C.M., & Smith, T.J. (May 2005). *Making progress
toward graduation: Evidence from the Talent Development High School
Model.* New York, NY: MDRC.

Levin, H.M., & McEwan, P.J. (2001). *Cost-effectiveness analysis:
Methods and Applications, 2nd ed*. Thousand Oaks: Sage.

Levine, A. (2005, March) *Educating School Leaders*. New York, NY:
Columbia University.

Malecki, C. K., & Elliot, S. N. (2002). Children's social behaviors as
predictors of academic achievement: A longitudinal analysis. *School
Psychology Quarterly, 17*, 1-23.

Monk, D. H. (1994). Subject area preparation of secondary mathematics
and science teachers Kane, T. (2004). Accessed on February 11, 2006 from
<http://www.wtgrantfoundation.org/usr_doc/After-school_paper.pdf>.

Murray, D. M. (1998). *Design and Analysis of Group-Randomized Trials*.
New York: Oxford University Press.

Najaka, S. S., Gottfredson, D. C., & Wilson, D. B. (2001). A
meta-analytic inquiry into the relationship between selected risk
factors and problem behavior. *Prevention Science, 2*, 257-271.

National Center for Education Statistics (2001). *Indicators of school
crime and safety*. Washington, DC: Office of Educational Research and
Improvement, U.S. Department of Education; Office of Justice Programs,
U.S. Department of Justice.

National Center for Education Statistics (NCES). (2003). *Remedial
education at degree-granting postsecondary institutions in Fall 2000*.
U.S. Department of Education. Washington, DC: U.S. Government Printing
Office.

National Research Council. (1999). *Improving student learning: A
strategic plan for education*

*Research and its utilization*. Committee on a Feasibility Study for a
Strategic Education

Research Program. Commission on Behavioral and Social Sciences and
Education.

Washington, DC: National Academy Press.

National Research Council. (2000). *How people learn: Brain, mind,
experience and school*.

Committee on Developments in the Science of Learning. J. Bransford, A.
Brown, and

R. Cocking (Eds.). Committee on Learning Research and Educational
Practice. S. Donovan,

J. Bransford, and J. Pellegrino (Eds.). Commission on Behavioral and
Social Sciences in

Education. Washington, DC: National Academy Press.

National Research Council (2002). *Scientific research in education*.
Committee on Scientific

Principles for Education Research. R.J. Shavelson and L. Towne (Eds.).
Center for

Education. Division of Behavioral and Social Sciences and Education.
Washington, DC:

National Academy Press.

Princiotta, D., Flanagan, K. D., and Germino Hausken, E. (2006). *Fifth
Grade: Findings From The Fifth-Grade Follow-up of the Early Childhood
Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K)*. (NCES
2006-038) U.S. Department of Education.

Sinclair, M. F., Christenson, S. L., Evelo, D. L., & Hurley, C. M.
(1998). Dropout prevention for high-risk youth with disabilities:
Efficacy of a sustained school engagement procedure. *Exceptional
Children, 65* (1), 7-21.

Snow, C. E., Burns, M. S., & Griffin, P. (Eds.), (1998). Preventing
reading difficulties in young children. Washington, DC: National Academy
of Sciences.

Waters, J. T., Marzano, R. J., & McNulty, B. A. (2003). *Balanced
leadership: What 30 years of research tells us about the effect of
leadership on student achievement*. Aurora, CO: Mid-continent Research
for Education and Learning.

Wentzel, K. R., (1993). Does being good make the grade? Social behavior
and academic competence in middle school. *Journal of Educational
Psychology, 85*, 357-364.

Wise, L. L., Harris, C. D., Brown, D. G., Becker, D. E., Sun, S., &
Coumbe, K. L. (2003). *California High School Exit Examination: Year 4
evaluation report. Prepared by the Human Resources Research Organization
(HumRRO) for the California Department of Education*. Sacramento, CA:
HumRRO.

[^1]: For additional information on describing procedures for
    randomization, see the What Works Clearinghouse document, *Evidence
    Standards for Reviewing Studies* (p. 6), available at
    <http://www.whatworks.ed.gov/reviewprocess/study_standards_final.pdf>.

[^2]: For more information, see Donner, A., & Klar, N. (2000). *Design
    and Analysis of Cluster Randomization Trials in Health Research*.
    New York: Oxford University Press; Murray, D. M. (1998). *Design and
    Analysis of Group-Randomized Trials*. New York: Oxford University
    Press; W.T. Grant Foundation & University of Michigan,
    <http://sitemaker.umich.edu/group-based/optimal_design_software>.

[^3]: For additional information on how to calculate the costs of a
    program or conduct an economic evaluation, applicants might refer to
    Levin, H.M., & McEwan, P.J. (2001). *Cost-Effectiveness Analysis*.
    2nd Ed. Thousand Oaks, CA: Sage Publications.