NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS SECONDARY ANALYSIS GRANTS

**CFDA NUMBER: 84.902B**

**RELEASE DATE: April 7, 2006**

**REQUEST FOR APPLICATIONS: IES-NCES-2007-01**

**INSTITUTE OF EDUCATION SCIENCES**

[**http://ies.ed.gov**](http://ies.ed.gov/)

**LETTER OF INTENT RECEIPT DATE: June 1, 2006**

**APPLICATION TRANSMITTAL DEADLINE: July 27, 2006**

**THIS REQUEST FOR APPLICATIONS CONTAINS THE FOLLOWING INFORMATION:**

# Section Page

**PART I [GUIDE TO THE RFA](#guide)** 3

## PART II GENERAL OVERVIEW

1\. [Request for Applications](#RFA) 4

2\. [Overview of the Institute\'s Research Programs](#ProgramsOverview)
4

A. [Outcomes](#Outcomes) 5

B. [Conditions](#Conditions) 5

C. [Grade Levels](#GradeLevels) 6

D. [Research Goals](#ResearchGoals) 6

## PART III RESEARCH GRANT TOPIC

# 3. [Purpose of the NAEP Secondary Data Analysis Program](#NAEPpurpose) 8

4\. [Background for the NAEP Secondary Data Analysis
Program](#Background) 8

#  

## PART IV REQUIREMENTS FOR PROPOSED RESEARCH

5\. [General Requirements of the Proposed Research](#GeneralReqs)

> A. [Basic Requirements](#BasicReqs) 10
>
> B. [Requirements for Goal One](#G1Reqs) (Identification Projects) 10

C. [Requirements for Goal Five](#G5Reqs) (Assessment and Measurement
Projects) 12

## PART V GENERAL SUBMISSION AND REVIEW INFORMATION

6\. [Applications Available](#AppsAvail) 15

7\. [Mechanism of Support](#MechSupport) 15

8\. [Funding Available](#FundsAvail) 15

# 9. [Eligible Applicants](#EligApps) 15

10\. [Special Requirements](#SpecReqs) 15

11\. [Letter of Intent](#LOI) 16

12\. [Submitting an Application](#Submitting) 16

13\. [Contents of Application](#Contents) 16

14\. [Application Processing](#AppProcessing) 19

15\. [Peer Review Process](#PeerReview) 20

16\. [Review Criteria for Scientific Merit](#ReviewCriteria) 20

17\. [Receipt and Start Date Schedule](#ReceiptSchedule) 21

18\. [Award Decisions](#AwardDecisions) 21

19\. [Inquiries May Be Sent To](#Inquiries) 21

20\. [Program Authority](#ProgramAuthority) 21

21\. [Applicable Regulations](#Regulations) 22

[]{#guide .anchor}**PART I GUIDE TO THE RFA**

*IES is presenting grant opportunities in a new format this year. To
make it as easy as possible and less time consuming for the
reader/applicant, this section identifies the major differences from
last year's format and describes the consequent organization of
information in this year's three (3) Requests For Applications (RFA's).*

In FY2006, the Institute of Education Sciences (IES) held a larger
number of formal grant competitions, each one addressing a distinct
topic area and each with its own RFA. Not only did the National Center
for Education Statistics (NCES) offer a National Assessment of
Educational Progress (NAEP) secondary analysis grant competition, there
were also separate RFA's for Reading and Writing, Mathematics and
Science Education, etc. Both the National Center for Education Research
(NCER) and the National Center for Special Education Research (NCSER)
offered multiple, single-topic competitions.

In FY2007, IES is holding fewer formal grant competitions but addressing
more topics. There are three competitions: this one, addressing NAEP
secondary analyses (through NCES), one addressing education research
(through NCER); and one addressing special education research (through
NCSER). The education and special education competitions each encompass
multiple, specific topic areas.

Last year each topic-specific RFA was self-contained. This year the NAEP
RFA remains self-contained. The NCER and NCSER RFA's are organized into
sections that contain information that is common to all topics and
sections that contain topic-specific information.

This RFA (IES-NCES-2007-01) describes the NAEP secondary analysis
research competition. Applications for this competition have an
application transmittal deadline of July 27, 2006, and will be reviewed
in the fall of 2006.

Also new this year are the forms for submitting applications
electronically.
[Highlights](http://www.ed.gov/about/offices/list/ies/highlights.doc) of
the forms will be available on the web no later than April 11, 2006.

Information on additional education research topics may be found in the
IES-NCER-2007-01 RFA, and information special education research topics
may be found in the IES-NCSER-2007-01 RFA. Topic-specific application
transmittal deadlines are specified within these RFA's.

**[Suggested options for reading this RFA:]{.underline}**

You may download the entire RFA as a .PDF file or you can navigate to
particular sections of the RFA on line.

Of course, this RFA may be read start to finish, or you may want to
start with Part III.

**PART II GENERAL OVERVIEW**

[]{#RFA .anchor}**[1. REQUEST FOR APPLICATIONS]{.underline}**

In this announcement, the Institute of Education Sciences (Institute)
describes the research program that is funded through its National
Center for Education Statistics. Separate announcements are available on
the Institute\'s website that pertain to discretionary grant
competitions funded through the Institute\'s National Center for
Education Research (<http://ies.ed.gov/ncer>) and National Center for
Special Education Research (<http://ies.ed.gov/ncser>).

The Institute invites applications to conduct secondary analyses of the
nationally representative achievement data collected by the National
Assessment of Educational Progress (NAEP) and the NAEP High School
Transcript Studies (HSTS). For this competition, the Institute will
consider only applications that meet the requirements outlined below
under the section on [General Requirements of the Proposed
Research](#GeneralReqs).

[]{#ProgramsOverview .anchor}**[2. OVERVIEW OF THE INSTITUTE\'S RESEARCH
PROGRAMS]{.underline}**

The Institute\'s over-arching priority is research that contributes to
improved academic achievement for all students, and particularly for
those whose education prospects are hindered by inadequate education
services and conditions associated with poverty, race/ethnicity, limited
English proficiency, disability, and family circumstance.

With academic achievement as the major priority, the Institute focuses
on outcomes that differ by periods of education. In the infancy and
preschool period, the outcomes of interest are those that enhance
readiness for schooling, for example, language skills, and for infants
and toddlers with disabilities, developmental outcomes. In kindergarten
through 12th grade, the core academic outcomes of reading and writing
(including reading and writing in the disciplines), mathematics, and
science are emphasized, as well as the behaviors and social skills that
support learning in school and successful transitions to employment,
independent living, and post-secondary education. At the post-secondary
level, the focus is on enrollment in and completion of programs that
prepare students for successful careers and lives. The same outcomes are
emphasized for students with disabilities across each of these periods,
and include the functional outcomes that improve educational and
transitional results. The acquisition of basic skills by adults with low
levels of education is also a priority.

In conducting research on academic outcomes, the Institute concentrates
on conditions within the control of the education system, with the aim
of identifying, developing, and validating effective education programs,
practices, policies, and approaches as well as understanding the factors
that influence variation in their effectiveness such as implementation.
Conditions that are of highest priority to the Institute are in the
areas of curriculum, instruction, assessment (including the
identification of students with disabilities), the quality of the
education workforce, and the systems and policies that affect these
conditions and their interrelationships (for example, accountability
systems, delivery mechanisms including technology, and policies that
support the ability of parents to improve educational results for their
children through such means as choice of education services and
provision of school-related learning opportunities in the home).

In this section, the Institute describes the overall framework for its
research grant programs. Specific information on the research topic
described in this announcement may be found in the section pertaining to
[Requirements for Proposed Research](#GeneralReqs).

The Institute addresses the educational needs of typically developing
students through its Education Research programs and the needs of
students with disabilities through its Special Education Research
programs. Both the Education Research and the Special Education Research
programs are organized by outcomes (e.g., reading, mathematics), type of
education condition (e.g., curriculum and instruction; teacher quality;
administration, systems, and policy), grade level, and research goals.

[]{#Outcomes .anchor}**A. Outcomes**

The Institute\'s research programs focus on improvement of the following
education outcomes: (a) readiness for schooling (pre-reading,
pre-writing, early mathematics and science knowledge and skills, and
social development); (b) academic outcomes in reading, writing,
mathematics, and science; (c) student behavior and social interactions
within schools that affect the learning of academic content; (d) skills
that support independent living for students with significant
disabilities; and (e) educational attainment (high school graduation,
enrollment in and completion of post-secondary education).

[]{#Conditions .anchor}**B. Conditions**

In general, each of the Institute\'s research programs focuses on a
particular type of condition (e.g., curriculum and instruction) that may
affect one or more of the outcomes listed previously (e.g., reading).
The Institute\'s research programs are listed below according to the
primary condition that is the focus of the program.

**a. *Curriculum and instruction*.** Several of the Institute\'s
programs focus on the development and evaluation of curricula and
instructional approaches. These programs include: (a) Research on
Reading and Writing; (b) Research on Mathematics and Science Education;
(c) Research on Preschool Curriculum Evaluation; (d) Research on Social
and Character Development; (e) Early Intervention, Early Childhood
Special Education, and Assessment for Young Children with Disabilities;
(f) Mathematics and Science Special Education Research; (g) Reading,
Writing, and Language Development Special Education Research; (h)
Secondary and Transition Services Special Education Research; (i)
Serious Behavior Disorders Special Education Research; (j) Autism
Spectrum Disorders; and (k) Response to Intervention.

**b**. ***Quality of the Education Workforce*.** A second condition that
affects student learning and achievement is the quality of teachers and
education leaders (e.g., principals, superintendents). The Institute
funds research on how to improve teacher quality through its programs on
(a) Research on Teacher Quality (b) Research on Education Leadership;
and (c) Research Grants Program on the Quality of Teachers and Other
Service Providers for Students with Disabilities.

**c. *Administration, systems, and policy*.** A third approach to
improving student outcomes is to identify systemic changes in the ways
in which schools and districts are led, organized, managed, and operated
that may be directly or indirectly linked to student outcomes. The
Institute takes this approach in its programs on (a) Research on
Education Policy, Finance, and Systems; (b) Education Research on High
School Reform; (c) Special Education Research on Individualized
Education Programs and Individualized Family Service Plans; and (d)
National Assessment of Educational Progress (NAEP) Secondary Analysis
Research Program.

Applicants should be aware that some of the Institute\'s programs cover
multiple conditions. For example, the following programs cover multiple
conditions: (a) Research on Cognition and Student Learning; (b) Research
on High School Reform; and (c) Special Education Research on
Individualized Education Programs and Individualized Family Service
Plans. In addition, the NAEP Secondary Analysis program funds projects
that cut across conditions (programs, practices, and policies) and types
of students (regular education and special education students).

[]{#GradeLevels .anchor}**C. Grade Levels**

The Institute\'s research programs also specify the ages or grade levels
covered in the research program. The specific grades vary across
research programs and within each research program, and grades may vary
across the research goals. In general, the Institute supports research
for (a) pre-kindergarten and kindergarten, (b) elementary school, (c)
middle school, (d) high school, (e) post-secondary education, (f)
vocational education, and (g) adult education. In addition, the
Institute supports research on infants with disabilities.

[]{#ResearchGoals .anchor}**D. Research Goals**

The Institute has established five research goals for its research
programs. Within each research program one or more of the goals may
apply: (a) Goal One -- identify existing programs, practices, and
policies that may have an impact on student outcomes and the factors
that may mediate or moderate the effects of these programs, practices,
and policies; (b) Goal Two -- develop programs, practices, and policies
that are theoretically and empirically based and obtain preliminary
(pilot) data on the relation (association) between implementation of the
program, practice, or policy and the intended education outcomes; (c)
Goal Three -- establish the efficacy of fully developed programs,
practices, or policies that either have evidence of a positive
correlation between implementation of the intervention and education
outcomes *or* are widely used but have not been rigorously evaluated;
(d) Goal Four -- provide evidence on the effectiveness of programs,
practices, and policies implemented at scale; and (e) Goal Five --
develop or validate data and measurement systems and tools. **Note: for
the NAEP secondary analysis research program described in this RFA, only
Goal One and Goal Five apply.**

For a list of the Institute\'s FY 2007 research grant topics --
including research grant competitions through the Institute\'s National
Center for Education Research, National Center for Special Education
Research, and National Center for Education Statistics, please see Table
1 below. This list includes the Postdoctoral Research Training
Fellowships in the Education Sciences, which is not a research grant
program. Funding announcements for these competitions may be downloaded
from the Institute\'s website at
[http://ies.ed.gov](http://ies.ed.gov/).

**[Table 1: FY 2007 Research Grant Topics:]{.underline}**

### **Education Research on**

1 Reading and Writing

2 Interventions for Struggling Adolescent and Adult Readers and Writers

3 Mathematics and Science Education

4 Teacher Quality -- Reading and Writing

5 Teacher Quality -- Mathematics and Science Education

## 6 Education Leadership

7 Education Policy, Finance, and Systems

8 Cognition and Student Learning

9 High School Reform

10 Postsecondary Education

11 Research Training Grants

# **Special Education Research on**

12 Early Intervention, Early Childhood Special Education, and Assessment
for Young Children with Disabilities

13 Individualized Education Programs and Individualized Family Service
Plans

14 Mathematics and Science Education

15 Reading, Writing, and Language Development

16 Secondary and Transition Services

17 Serious Behavior Disorders

18 Quality of Teachers and Other Service Providers for Students with
Disabilities

19 Autism Spectrum Disorders

20 Response to Intervention

21 Assessment for Accountability

**[National Assessment of Education Progress]{.underline}**

#### 22 Secondary Analysis of Data from the National Assessment of Educational Progress 

## PART III RESEARCH GRANT TOPIC

[]{#NAEPpurpose .anchor}**3.** **PURPOSE OF THE NAEP SECONDARY ANALYSIS
PROGRAM**

Through its NAEP secondary analysis program, the Institute intends to
contribute to improvement of student learning and achievement by (a)
identifying programs, policies, and practices that are potentially
effective for improving academic outcomes, as well as mediators and
moderators of the effects of these programs, policies, and practices,
and (b) developing tools or procedures to assist NAEP users in the
analysis, interpretation and reporting of state- and district-level NAEP
results or to improve precision in the estimation and reporting of NAEP
results.

[]{#Background .anchor}**4. BACKGROUND FOR THE NAEP SECONDARY ANALYSIS
PROGRAM**

Mandated by Congress, the National Assessment of Educational Progress
(NAEP) surveys the education achievement of students in the United
States, and monitors their progress over time. Widely known as the
"Nation's Report Card," NAEP has been collecting data to provide
educators and policymakers valid and meaningful information for more
than 30 years.

The NAEP program includes two distinct components: "main NAEP" and
"long-term trend NAEP." These two components use distinct data
collection procedures, separate samples of students defined by different
criteria, and different test instruments based on different frameworks.
Main NAEP includes assessment instruments based on frameworks typically
developed within the past 10 years. Results from the main NAEP
assessments are reported at the national and, in some subjects, at the
state level. In 2002 and 2003, exploratory assessments were conducted in
a small group of large, urban school districts to determine the
feasibility of reporting assessment results at the district level as
well. The subject areas assessed as part of the main assessments include
reading, mathematics, science, writing, U.S. history, geography, civics,
and the arts. State-level results, however, are reported only in
reading, mathematics, science, and writing. Background data collected
through questionnaires administered to sampled students, the teachers of
those students, and administrators of the schools those students attend
are also available for main NAEP assessments. The long-term trend
assessments are conducted at the national level only, and are
administered less frequently than the main assessments. Long-term trend
assessments are conducted in only reading and mathematics. Background
data for schools and students are also available for the long-term trend
assessment.

In addition to these assessment programs, NAEP periodically conducts the
High School Transcript Study (HSTS) to investigate the current course
offerings and course-taking patterns in the nation's secondary schools.
Thousands of transcripts of high school seniors who graduate from public
and nonpublic high schools are collected from a nationally
representative sample of schools. Transcript study data are linked to
the NAEP 12th grade assessment results providing information on the
relationship between course-taking patterns and achievement.

NAEP produces a number of different publications each time assessment
results are released. These reports provide summary data to the general
public, and focus on the overall national and state results, as well as
subgroups of the population. In addition, NAEP has an extensive web site
([*http://nces.ed.gov/nationsreportcard*](http://nces.ed.gov/nationsreportcard))
that provides easy access to all NAEP publications. The web site also
houses a number of tools that make the NAEP data and released assessment
questions accessible to the general public.

The data collected by NAEP represents one of the richest and most
in-depth databases of information about student achievement. As part of
its mission to the education community, NAEP encourages researchers and
policy makers to make use of the data and to perform their own analyses
and studies on education achievement. Beyond the summary data available
on the NAEP website and in NAEP reports, complete access to all detailed
data is available to data analysts who apply for and are granted
restricted use licenses.

Despite the depth of the information and the availability of the
databases, the Institute believes that the NAEP database remains
underutilized. Through this program of secondary analysis grants, the
Institute hopes to encourage greater use of the NAEP data to inform
education research, policy, and practice. Much potentially valuable
information that could be gained from the NAEP data remains untapped.
This grant program was developed to make resources available to
qualified data analysts to explore the NAEP data more fully.

By broadening the user base, the Institute believes that not only will
the data be more widely disseminated, but also fresh perspectives and
new ideas will be applied to analysis of NAEP data. Although the federal
government assumes responsibility for collecting these data and making
them available to the public, there are opportunities for more analysis
of and reporting on the NAEP data than can or should be done by the
federal government. By encouraging such broad use of this rich database,
the Institute expects that education policy and practice can be informed
and enhanced.

It is also expected that by inviting data analysts to work more closely
with the NAEP assessment and its data, the program will benefit from
additional perspectives on the strengths and weakness of the current
methodology underlying NAEP. The Institute welcomes applications for
studies that will explore new methodological techniques and new software
or analysis models that may help make the NAEP data accessible to a
broader range of users.

Descriptions of recently funded proposals are available at
<http://nces.ed.gov/nationsreportcard/researchcenter/naepgrants.asp>.

###### PART IV REQUIREMENTS FOR PROPOSED RESEARCH

[]{#GeneralReqs .anchor}**[5. GENERAL REQUIREMENTS OF THE PROPOSED
RESEARCH]{.underline}**

[]{#BasicReqs .anchor}**A. Basic Requirements**

**a. *Resubmissions*.** Applicants who intend to revise and resubmit a
proposal that was submitted to one of the Institute's FY2006
competitions but that was not funded must indicate on the application
form that their FY 2007 proposal is a revised proposal. Their FY 2006
reviews will be sent to this year\'s reviewers along with their
proposal. Applicants should indicate the revisions that were made to the
proposal on the basis of the prior reviews using no more than 3 pages of
Appendix A.

**b. *Applying to multiple competitions or topics.*** Applicants may
submit proposals to more than one of the Institute\'s FY 2007
competitions or topics. In addition, within a particular competition or
topic, applicants may submit multiple proposals. However, applicants may
submit a given proposal only once (i.e., applicants may not submit the
same proposal or very similar proposals to multiple topics or to
multiple goals in the same topic or to multiple competitions). If the
Institute determines prior to panel review that an applicant has
submitted the same proposal or very similar proposals to multiple topics
within or across competitions and the proposal is judged to be compliant
and responsive to the submission rules and requirements described in the
Request for Applications, the Institute will select one version of the
application to be reviewed by the appropriate scientific review panel.
If the Institute determines after panel review that an applicant has
submitted the same proposal or very similar proposals to multiple topics
within or across competitions and if the proposal is determined to be
worthy of funding, the Institute will select the topic under which the
proposal will be funded.

**c. *Applying to a particular goal within a competition*.** To submit
an application to one of the Institute\'s education research programs,
applicants must choose the specific goal under which they are applying.
Each goal has specific requirements.

d.  ***Inclusions and restrictions on interventions under each
    competition*.** For the NAEP Secondary Analysis competition,
    applicants must propose analyses using at least one of the currently
    available NAEP or HSTS data sets. Applicants who are interested in
    conducting research projects that generate new data on education
    processes and attainment should review the other research grant
    programs supported by the Institute
    ([http://ies.ed.gov](http://ies.ed.gov/)).

**For the FY 2007 NAEP Secondary Analysis competition, applicants must
submit under *either* Goal One *or* Goal Five. The numbering of goals is
consistent across the Institute\'s research programs. The NAEP secondary
analysis program only supports Goals One and Five.**

[]{#G1Reqs .anchor}**B. Requirements for Goal One (Identification)**

**a. *Purpose of identification studies*.** Through all of its research
programs that include the Identification goal (Goal One), the Institute
is primarily interested in analyses of multivariate data, such as
longitudinal individual student data that exist in a number of federal-,
state-, and district-level databases, to identify existing programs,
practices, and policies that may be associated with better academic
outcomes, and to examine factors and conditions that may mediate or
moderate the relations between student outcomes and these programs,
practices, and policies.

Under Goal One of the NAEP secondary analysis program, the Institute
expects applicants to use at least one of the currently existing NAEP
data sets to capitalize on natural variation in education practices or
policies. For example, investigators might link data on state teacher
certification requirements that have been compiled by the investigators
with NAEP Grade 4 reading and mathematics achievement data to examine
the relations between different requirements and student achievement.

The strongest approaches to statistical modeling of multivariate data
involve testing two or more models of relationships using the same data.
Because multivariate analyses cannot fully adjust for selection biases
and the effects of variables that were not measured or were not measured
well, they are seldom if ever sufficient to support strong causal
conclusions about what works. However, when two or more models of
relationships among variables are tested with the same data, it may be
possible to determine that one is more plausible than another, thus
providing information relevant to understanding what does not work, as
well as what does work. That, in turn, can direct future efforts in
avenues that are more likely to be productive.

Evidence obtained through a Goal One project of the association between
exposure to a program, practice, or policy and better student outcomes
has the possibility of being used to support a subsequent application
for a Goal Two (Development) or Goal Three (Efficacy) project.

**For the NAEP secondary analysis program, applicants must use at least
one of the currently existing NAEP data sets.**

*By addressing the theoretical and empirical rationale for the study and
the practical importance of the intervention (e.g., policy, program)
that will be examined,* *Goal One applicants are addressing aspects of
the significance of their proposal.*

**b. *Methodological requirements*. For all applications, including
those submitted under Goal One, the proposed research design must be
appropriate for answering the research questions or hypotheses that are
posed.**

[]{.mark}

\(i\) *Database*. The applicant should describe clearly the database(s)
to be used in the investigation including information on sample
characteristics, variables to be used, and ability to ensure access to
the database if the applicant does not already have access to it. The
database should be described in sufficient detail so that reviewers will
be able to judge whether or not the proposed analyses may be conducted
with the database. If multiple databases will be linked to conduct
analyses, applicants should provide sufficient detail for reviewers to
be able to judge the feasibility of the plan.

\(ii\) *Data analysis.* The applicant must include detailed descriptions
of data analysis procedures. The data analytic strategy used in the
analysis of any NAEP data must take into account the sampling and the
psychometric designs of the NAEP data. All the NAEP data are collected
using a multi-stage clustered sampling design. The sampling designs for
the national data and most of the older assessment data sets also use
stratification and over-sampling within strata. This sampling design has
major implications for secondary analysis of the NAEP data.

Similarly, the NAEP psychometric design produces proficiency estimates
that are not actual individual scores for the students who participate
in the assessments. These multiply imputed estimates of student
proficiency are called "plausible values," and five such values
constitute the "score" for each examinee. Secondary analysts may use
these plausible values as their unit of analysis or they may use
marginal maximum likelihood techniques to estimate the statistics of
interest to their study directly. The psychometric design of NAEP also
has implications for secondary analysis.

An accurate and defensible explanation of how the proposed project will
account for both the sampling and psychometric design of NAEP is a
critical component of every application to this program.

Because predictor variables relevant to education outcomes (e.g.,
student characteristics, teacher characteristics, school and district
characteristics) often co-vary, the Institute expects investigators to
utilize the most appropriate state-of-the-art analytic techniques to
isolate the possible effects of variables of interest. Analytic
strategies should allow investigators to examine mediators and
moderators of programs and practices. The relation between hypotheses,
measures, independent and dependent variables should be well specified.
Strong applications will include an explicit discussion of how exclusion
from testing, or missing data, will be handled within the statistical
analyses. Strong applications will propose an approach for comparing
hypotheses or models of relationships among variables.

**c. *Personnel and resources*.** Competitive applicants will have
research teams that collectively demonstrate expertise in (a) the
relevant academic content area (e.g., reading, mathematics); and (b)
implementation of and analysis of results from the research design that
will be employed. Competitive applicants will have access to
institutional resources that adequately support research.

**d. *Awards*.** Typical awards for projects under this goal will be
\$65,000 to a maximum of \$100,000 (total cost = direct cost + indirect
cost) for a period not to exceed 18 months. Applications for smaller
awards and shorter durations are also welcome. The size of the award
depends on the scope of the project.

[]{#G5Reqs .anchor}**C. Applications under Goal Five (Measurement)**

Across the Institute\'s research programs, the Measurement goals differ
in purpose. Requirements described below apply to the NAEP secondary
analysis research program.

[]{.mark}

**a. *Purpose of NAEP Goal Five proposals***. Under Goal Five, the
Institute intends to improve the usefulness of NAEP data. Applicants
under Goal Five must propose projects to develop tools or procedures
that will assist other users of the NAEP data to analyze, interpret and
report NAEP data more easily and accurately or may propose projects to
develop methodological or analytic procedures that will improve
precision in the estimation and reporting of NAEP results.

An example of an appropriate project under Goal Five is one that adapts
an existing system for creating student skill profiles for use with the
NAEP data. Student skill profiles would allow NAEP results to be
reported in terms of mastery and non-mastery of the skills represented
in the NAEP achievement levels. Applicants under Goal Five may propose
projects that are of particular interest to their state education agency
or professional organization and that can be generalized to be useful to
other NAEP users. Projects that include the development of new software
that permits advanced analytic techniques to be readily applied to the
NAEP data are encouraged under this goal.

Also appropriate under Goal Five are robustness studies or validity
studies. Applicants may propose to test alternatives to some component
of the NAEP sampling or psychometric model or to test analytic solutions
to problems that were previously intractable in the context of NAEP. For
example, an applicant might propose to compare different approaches to
estimating statistical bias in analyses of the NAEP data.

**For the NAEP secondary analysis program, applicants must use at least
one of the currently existing NAEP data sets.**

**b. *Requirements of proposed tools or procedures.*** Applicants should
provide a compelling rationale for the proposed tool or procedure.
Reviewers will consider the strength of the theoretical foundation for
the proposed tool or procedure, the existing empirical evidence
supporting the proposed tool or procedure, whether the proposed tool or
procedure duplicates existing tools or procedures, and whether the tool
or procedure is practical for potential users.

By describing the theoretical and empirical support for and the
practical utility of the proposed tool or procedure, applicants are
addressing aspects of the significance of their proposal.

**c. *Methodological requirements***.

\(i\) *Database*. The applicant should describe clearly the database(s)
to be used in the investigation including information on sample
characteristics, variables to be used, and ability to ensure access to
the database if the applicant does not already have access to it. The
database should be described in sufficient detail so that reviewers will
be able to judge whether or not the proposed analyses may be conducted
with the database. If multiple databases will be linked to conduct
analyses, applicants should provide sufficient detail for reviewers to
be able to judge the feasibility of the plan.

\(ii\) *Data analysis.* The applicant must include detailed descriptions
of data analysis procedures. The data analytic strategy used in the
analysis of any NAEP data must take into account the sampling and the
psychometric designs of the NAEP data. All the NAEP data are collected
using a multi-stage, clustered sampling design. The sampling designs for
the national data and most of the older assessment data sets also use
stratification and over-sampling within strata. This sampling design has
major implications for secondary analysis of the NAEP data.

> Similarly, the NAEP psychometric design produces proficiency estimates
> that are not actual individual scores for the students who participate
> in the assessments. These multiply imputed estimates of student
> proficiency are called "plausible values," and five such values
> constitute the "score" for each examinee. Secondary analysts may use
> these plausible values as their unit of analysis or they may use
> marginal maximum likelihood techniques to estimate the statistics of
> interest to their study directly. The psychometric design of NAEP also
> has implications for secondary analysis.

An accurate and defensible explanation of how the proposed project will
account for both the sampling and psychometric design of NAEP is a
critical component of every application to this program.

> The Institute expects investigators to utilize the most appropriate
> state-of-the-art analytic techniques to investigate the research
> questions of interest. The relation between hypotheses, measures,
> independent and dependent variables should be well specified. Strong
> applications will include an explicit discussion of how exclusion from
> testing, or missing data, will be handled within the statistical
> analyses.

**d. *Personnel and resources****.* Competitive applicants will have
research teams that collectively demonstrate expertise in (a) the
relevant academic content area (e.g., reading, mathematics); and (b)
implementation of and analysis of results from the research design that
will be employed. Competitive applicants will have access to
institutional resources that adequately support research.

**e. *Awards*.** Typical awards for projects under this goal will be
\$65,000 to a maximum of \$100,000 (total cost = direct cost + indirect
cost) for a period not to exceed 18 months. Applications for smaller
awards and shorter durations are also welcome. The size of the award
depends on the scope of the project.

##### PART V GENERAL SUBMISSION AND REIVIEW INFORMATION

[]{#AppsAvail .anchor}**[6. APPLICATIONS AVAILABLE]{.underline}**

Application forms and instructions for the electronic submission of
applications will be available for the program of research listed in
this RFA from the following web site:

[https://ies.constellagroup.com](https://ies.constellagroup.com/)

by the following date:

NAEP Secondary Analysis Program June 15, 2006

[]{.mark}

The application form approved for use in the competitions specified in
this RFA is the new, government-wide SF424 Research and Related (R&R)
Form (OMB Number 4040-0001).

[]{#MechSupport .anchor}**[7. MECHANISM OF SUPPORT]{.underline}**

The Institute intends to award grants for periods up to 18 months
pursuant to this request for applications. Please see specific details
for each goal in the [General Requirements of the Proposed
Research](#GeneralReqs) section of the announcement.

[]{#FundsAvail .anchor}**[8. FUNDING AVAILABLE]{.underline}**

The size of the award depends on the scope of the project. Please see
specific details in the [General Requirements of the Proposed
Research](#GeneralReqs) section of the announcement. Although the plans
of the Institute include this program of research, awards pursuant to
this request for applications are contingent upon the availability of
funds and the receipt of a sufficient number of meritorious
applications. The number of projects funded under a specific goal
depends upon the number of high quality applications submitted to that
goal. The Institute does not have plans to award a specific number of
grants under each particular goal.

[]{#EligApps .anchor}**[9. ELIGIBLE APPLICANTS]{.underline}**

Because only organizations may be licensed to receive the restricted use
NAEP data, only organizations may apply for grants under this program.
Any organization or consortium of organizations that has the knowledge
and capacity to conduct secondary analysis of the NAEP data is eligible
to apply. Eligible applicants include, but are not limited to,
non-profit and for-profit organizations and public and private agencies
and institutions, such as colleges and universities.

[]{#SpecReqs .anchor}**[10. SPECIAL REQUIREMENTS]{.underline}**

Research supported through this program must be relevant to U.S.
schools.

Recipients of awards are expected to publish or otherwise make publicly
available the results of the work supported through this program. The
Institute asks IES-funded investigators to submit voluntarily to the
Educational Resources Information Center (ERIC) an electronic version of
the author\'s final manuscript upon acceptance for publication in a
peer-reviewed journal, resulting from research supported in whole or in
part, with direct costs from the Institute. The author\'s final
manuscript is defined as the final version accepted for journal
publication, and includes all modifications from the peer review
process. Details of the Institute\'s policy are posted on the
Institute\'s website at [http://ies.ed.gov]{.underline}.

Research applicants may collaborate with, or be, for-profit entities
that develop, distribute, or otherwise market education products or
services. Involvement of the developer or distributor must not
jeopardize the objectivity of the research.

[]{#LOI .anchor}**[11. LETTER OF INTENT]{.underline}**

A letter indicating a potential applicant's intent to submit an
application is optional, but encouraged, for each application. The
letter of intent must be submitted electronically by the date listed at
the beginning of this document, using the instructions provided at the
following web site:

<https://ies.constellagroup.com/>

The letter of intent should include a descriptive title, the goal which
the application will address, and brief description of the research
project (about 3,500 characters including spaces, which is approximately
one page, single-spaced); the name, institutional affiliation, address,
telephone number and e-mail address of the principal investigator(s);
and the name and institutional affiliation of any key collaborators. The
letter of intent should indicate the duration of the proposed project
and provide an estimated budget request by year, and a total budget
request. Although the letter of intent is optional, is not binding, and
does not enter into the review of subsequent applications, the
information that it contains allows Institute staff to estimate the
potential workload to plan the review.

[]{#Submitting .anchor}**[12. SUBMITTING AN APPLICATION]{.underline}**

Applications must be submitted **electronically by 8:00 p.m**. **Eastern
time** by the application transmittal deadline, using the standard forms
and the instructions provided at the following web site:

[https://ies.constellagroup.com](https://ies.constellagroup.com/)

Potential applicants should check this site for information about the
electronic submission procedures that must be followed and the software
that will be required.

[]{#Contents .anchor}**[13.]{.underline}** **[CONTENTS OF
APPLICATION]{.underline}**

All applications and proposals for Institute funding must be
self-contained within specified page limitations. Internet Web site
addresses (URLs) may not be used to provide information necessary to the
review because reviewers are under no obligation to view the Internet
sites.

All of the instructions and requirements regarding (a) submission of the
application, (b) acceptable format of the application, (c) page
limitations, and (d) required forms will be provided on the application
submission website
([https://ies.constellagroup.com](https://ies.constellagroup.com/)).

In this section, the Institute provides instructions regarding the
content of the (a) project summary/abstract, (b) project narrative, (c)
bibliography and references cited, (d) biographical sketches of key
project personnel, (e) narrative budget justification, (f) subaward
budget, (g) Appendix A, and (h) additional forms.

**A. Project Summary/Abstract**

The project summary/abstract is limited to 1 single-spaced page and must
adhere to the margin, format, and font size requirements described in
the project narrative section. The project summary/abstract will be
submitted as a .PDF attachment, and should include: (1) The title of the
project; (2) the RFA goal under which the applicant is applying (e.g.,
identification, measurement); and brief descriptions of (3) the purpose
(e.g., to develop and obtain preliminary evidence of potential efficacy
of a reading comprehension intervention for struggling high school
readers); (4) the sample population(s) (age groups, race/ethnicity,
SES); (5) key variables; and (6) data analytic strategy (e.g.,
structural equation models, hierarchical linear models).

**B. Project Narrative**

The project narrative will be submitted as a .PDF attachment.
Incorporating the requirements outlined under the section on
[Requirements of the Proposed Research](#GeneralReqs), the project
narrative provides the majority of the information on which reviewers
will evaluate the proposal. The project narrative must include the four
sections described below (a. \"Significance\" through d. \"Resources\")
in the order listed and must conform to the format requirements
described on the application submission website.

The project narrative is limited to **20 single-spaced pages** for all
applicants. The 20-page limit for the project narrative does not include
any of the SF 424 forms, the one-page summary/abstract, the appendices,
research on human subjects information, bibliography and references
cited, biographical sketches of senior/key personnel, narrative budget
justification, sub award budget information or certifications and
assurances. **Reviewers are able to conduct the highest quality review
when applications are concise and easy to read, with pages numbered
consecutively**.

For the purposes of applications submitted under this RFA, a "page" is
8.5 in. x 11 in., on one side only, with 1 inch margins at the top,
bottom, and both sides. Text must be single spaced in the narrative. To
ensure that the text is easy for reviewers to read and that all
applicants have the same amount of available space in which to describe
their projects, applicants must adhere to the type size and format
specifications for the entire narrative including footnotes. **It is
very important that applicants review carefully the "Application Format
Requirements" outlined in [*Fiscal Year 2007 Application Package
Highlights*](http://www.ed.gov/about/offices/list/ies/highlights.doc).**

**a. *Significance****.* Describe the contribution the study will make
to providing a solution to an education problem identified in the
Background Section of this RFA.

Provide a compelling rationale addressing, where applicable, the
theoretical foundation, relevant prior empirical evidence, and the
practical importance of the proposed project.

**b.** *Methods.* The Methods section of the application should address
all of the requirements detailed in the methodological requirements
sections for [Goal One](#G1Reqs) and [Goal Five](#G5Reqs)

\(i\) Include clear, concise hypotheses or research questions;

\(ii\) Present a clear description of, and a rationale for, the
assessment or transcript study data selected; and

\(ii\) Present a detailed data analysis plan that justifies and explains
the selected analysis strategy, shows clearly how the measures and
analyses relate to the hypotheses or research questions, and indicates
how the results will be interpreted. Applicants must account for the
sampling and psychometric constraints of the NAEP data.

**c. *Personnel.*** Include brief descriptions of the qualifications of
key personnel (information on personnel should also be provided in their
curriculum vitae). For each of the key personnel, please describe the
roles, responsibilities, and percent of time devoted to the project.

**d. *Resources*.** Provide a description of the resources available to
support the project at the applicant's institution and in the field
settings in which the research will be conducted.

**C. Bibliography and References Cited**

This section will be submitted as a .PDF attachment. Please include
complete citations, including titles and all authors, for literature
cited in the research narrative.

**D. Biographical Sketches of Senior/Key Personnel**

This section will be submitted as a .PDF attachment. Abbreviated
curriculum vitae should be provided for the principal investigator(s)
and other key personnel. *Each vita is limited to 4 pages and should
include information sufficient to demonstrate that personnel possess
training and expertise commensurate with their duties (e.g.,
publications, grants, relevant research experience) and have adequate
time devoted to the project to carry out their duties (e.g., list
current and pending grants with the proportion of the individual\'s time
allocated to each project).* Previous experience with NAEP data should
be emphasized. The curriculum vita must adhere to the margin, format,
and font size requirements described in the project narrative section.

## E. Narrative Budget Justification 

This section will be submitted as a .PDF attachment and should provide
sufficient detail to allow reviewers to judge whether reasonable costs
have been attributed to the project. The budget justification should
correspond to the itemized breakdown of project costs that is provided
in the Research & Related Budget (SF 424) Sections A & B; C, D, &E; and
F-K. It should include the time commitments and brief descriptions of
the responsibilities of key personnel. For consultants, the narrative
should include the number of days of anticipated consultation, the
expected rate of compensation, travel, per diem, and other related
costs. A justification for equipment purchase, supplies, travel and
other related project costs should also be provided in the budget
narrative for each project year outlined in the Research & Related
Budget (SF 424).

For those applications that include a subaward(s) for work conducted at
collaborating institutions, the narrative should also provide the
details about the subaward(s). Include the actual subaward budgets as a
separate attachment. (See below "[Subaward Budget](#subawardbudget)".)

[]{.mark}

Applicants should use their institution's federal indirect cost rate and
use the off-campus indirect cost rate where appropriate (see
instructions under Section 9 Special Requirements). If less than 75
percent of total indirect costs are based on application of the
off-campus rate, the applicant should provide a detailed justification.

[]{#subawardbudget .anchor}**F. Subaward Budget**

This section will be submitted as a .PDF attachment. For applications
that include a subaward(s) for work conducted at collaborating
institutions, applicants must submit an itemized budget spreadsheet for
each subaward for each project year. As noted above, the details of the
subaward costs should be included in the Narrative Budget Justification.
An Excel spreadsheet will be provided in the electronic application
package to allow applicants to enter the subaward budget information in
accordance with the prescribed format. Applicants will complete the
spreadsheet in Excel format, convert it to a .PDF file, and then upload
it as an attachment.

**G. Appendix A**

Appendix A should be included at the end of the Project Narrative, and
will be submitted as part of the same .PDF attachment.

The purpose of A*ppendix A* is to allow the applicant to include any
figures, charts, or tables that supplement the research text, examples
of measures to be used in the project, and letters of agreement from
partners (e.g., schools) and consultants. In addition, in the case of a
resubmission, the applicant may use **up to 3 pages** of the appendix to
describe the ways in which the revised proposal is responsive to prior
reviewer feedback. These are the only materials that may be included in
Appendix A; all other materials will be removed prior to review of the
application. Narrative text related to any aspect of the project (e.g.,
descriptions of the proposed sample, the design of the study, or
previous research conducted by the applicant) must be included in the
research or postdoctoral training narrative. Letters of agreement should
include enough information to make it clear that the author of the
letter understands the nature of the commitment of time, space, and
resources to the research project that will be required if the
application is funded. The appendix is limited to 15 pages. The
Institute recognizes that some applicants may have more letters of
agreement than will be accommodated by the 15-page limit. In such
instances, applicants should include the most important letters of
agreement and may list the letters of agreement that are not included in
the application due to page limitations.

**H. Additional Forms**

Please note that applicants selected for funding will be required to
submit the following certifications and assurances before a grant is
issued:

> \(1\) SF 424B-Assurances-Non-Construction Programs
>
> \(2\) ED-80-0013-Certification Regarding Lobbying, Debarment,
> Suspension and other Responsibility Matters; and Drug-Free Workplace
> Requirements
>
> \(3\) ED 80-0014 (if applicable)-Lower Tier Certification
>
> \(4\) SF-LLL (if applicable) - Disclosure of Lobbying Activities
>
> \(5\) Protection of Human Research Subjects assurance and/or
> Institutional Review Board certification, as appropriate

[]{#AppProcessing .anchor}**[14. APPLICATION PROCESSING]{.underline}**

Applications must be received by **8:00 p.m. Eastern time** on the
application transmittal deadline listed in the heading of this request
for applications. Upon receipt, each application will be reviewed for
compliance and for responsiveness to this request for applications.
Applications that do not address specific requirements of this request
will be returned to the applicants without further consideration.

[]{#PeerReview .anchor}**[15. PEER REVIEW PROCESS]{.underline}**

Applications that are compliant and responsive to this request will be
evaluated for scientific and technical merit. Reviews will be conducted
in accordance with the review criteria stated below by a panel of
scientists who have substantive and methodological expertise appropriate
to the program of research and request for applications.

Each application will be assigned to one of the Institute\'s scientific
review panels. At least two primary reviewers will complete written
evaluations of the application, identifying strengths and weaknesses
related to each of the review criteria. Primary reviewers will
independently assign a score for each criterion, as well as an overall
score, for each application they review. Based on the overall scores
assigned by primary reviewers, an average overall score for each
application will be calculated and a preliminary rank order of
applications prepared before the full peer review panel convenes to
complete the review of applications.

The full panel will consider and score only those applications deemed to
be the most competitive and to have the highest merit, as reflected by
the preliminary rank order. A panel member may nominate for
consideration by the full panel any proposal that he or she believes
merits full panel review but would not have been included in the full
panel meeting based on its preliminary rank order.

[]{#ReviewCriteria .anchor}**[16. REVIEW CRITERIA FOR SCIENTIFIC
MERIT]{.underline}**

The goal of Institute-supported research is to contribute to the
solution of education problems and to provide reliable information about
the education practices that support learning and improve academic
achievement and access to education for all students. Reviewers for all
applications will be expected to assess the following aspects of an
application in order to judge the likelihood that the proposed research
will have a substantial impact on the pursuit of that goal. Information
pertinent to each of these criteria is also described above in the
section on [Requirements of the Proposed Research](#GeneralReqs) and in
the description of the project narrative, which appears in the section
on [Contents of Application](#Contents).

**a. *Significance*.** Does the applicant present a compelling rationale
for the proposed project? Are there strong theoretical reasons,
empirical support, and practical reasons to justify the development
and/or evaluation of the proposed intervention or assessment? Does the
applicant make a compelling case for the potential contribution of the
project to the solution of an education problem?

**b. *Research plan*.** Does the applicant present (a) clear hypotheses
or research questions; (b) clear descriptions of and strong rationales
for the assessment or transcript data selected; and (c) a detailed and
well-justified data analysis plan? Does the research plan meet the
requirements described in the section on the Requirements of the
Proposed Research and in the description of the research narrative in
the section on Contents of Application? Is the research plan appropriate
for answering the research questions or testing the proposed hypotheses?

**c. *Personnel*.** Does the description of the personnel make it
apparent that the principal investigator, project director, and other
key personnel possess the training and experience and will commit
sufficient time to competently implement the proposed research?

**d. *Resources*.** Does the applicant have the facilities, equipment,
supplies, and other resources required to support the proposed
activities? Do the commitments of each partner show support for the
implementation and success of the project?

[]{#ReceiptSchedule .anchor}**[17. RECEIPT AND START DATE
SCHEDULE]{.underline}**

**A. Letter of Intent Receipt Date:**

NAEP Secondary Analysis applications June 1, 2006

**B.** **Application Transmittal Deadline:**

NAEP Secondary Analysis applications July 27, 2006

**C.** **Earliest Anticipated Start Date:**

NAEP Secondary Analysis applications March 1, 2007

[]{#AwardDecisions .anchor}**[18. AWARD DECISIONS]{.underline}**

The following will be considered in making award decisions:

-   Scientific merit as determined by peer review

-   Responsiveness to the requirements of this request

-   Performance and use of funds under a previous Federal award

-   Contribution to the overall program of research described in this
    request

-   Availability of funds

[]{#Inquiries .anchor}**[19. INQUIRIES MAY BE SENT TO:]{.underline}**

Dr. Alexandra Sedlacek

Institute of Education Sciences

National Center for Education Statistics

1990 K Street, NW

Washington, DC 20006

Email: <Alex.Sedlacek@ed.gov>

Telephone: (202) 502-7446

[]{#ProgramAuthority .anchor}

**[20. PROGRAM AUTHORITY]{.underline}**

20 U.S.C. 9010 [et]{.underline} [seq]{.underline}., section 303 of the
National Assessment of Educational Progress Authorization Act. This
program is not subject to the intergovernmental review requirements of
Executive Order 12372.

[]{#Regulations .anchor}

**[21. APPLICABLE REGULATIONS]{.underline}**

The Education Department General Administrative Regulations (EDGAR) in
34 CFR parts 74, 77, 80, 81, 82, 84, 85, 86 (part 86 applies only to
institutions of higher education), 97, 98, and 99. In addition 34 CFR
part 75 is applicable, except for the provisions in 34 CFR 75.100,
75.101(b), 75.102, 75.103, 75.105, 75.109(a), 75.200, 75.201, 75.209,
75.210, 75.211, 75.217, 75.219, 75.220, 75.221, 75.222, and 75.230.