# Biennial Report to Congress

# Institute of Education Sciences

The Education Sciences Reform Act of 2002 (ESRA) established a new
organization within the U.S. Department of Education, the Institute of
Education Sciences (the Institute). The mission of the Institute is to
expand knowledge and provide information on: the condition of education
(through the National Center for Education Statistics); practices that
improve education outcomes (through the National Center for Education
Research); the effectiveness of federal and other education programs
(through the National Center for Education Evaluation and Regional
Assistance); and the educational needs of children with disabilities
(through the National Center for Special Education Research as
authorized in December of 2004 in the Individuals with Disabilities
Education Improvement Act).

The Institute is led by a Director at the deputy secretary level who is
nominated by the President, confirmed by the Senate, and serves a term
of 6 years. Dr. Grover J. (Russ) Whitehurst, a research psychologist,
became the first director on November 22, 2002.

This bill creates an Institute of Education Sciences \... Of all of the
potential federal functions, the research role remains the most
unrealized in the course of the past half century, despite being one of
the few roles that almost everyone agrees is uniquely appropriate for
the federal government.

Christopher T. Cross,

Center for Education Policy

The Institute is overseen by the National Board for Education
Sciences---15 highly qualified individuals, the majority with strong
research backgrounds, who are nominated by the President and confirmed
by the Senate. The Board was confirmed by the Senate at the close of the
108^th^ Congress, and held its first meeting in February of 2005. The
Institute has a full-time staff complement of about 200 and is
responsible for roughly \$600 million annually in external grants and
contracts through six budget line items (research and dissemination,
statistics, assessment, special education research, regional
laboratories, and statewide data systems) and through evaluation
set-asides in the budgets of other Department of Education programs.

The Institute was born out of a shared sense among policymakers, the
legislative branch, and the executive branch that education practice and
research were badly in need of reform. Capturing this view, the National
Research Council concluded that, "the complex world of
education---unlike defense, health care, or industrial production---does
not rest on a strong research base. In no other field are personal
experience and ideology so frequently relied on to make policy choices,
and in no other field is the research base so inadequate and little
used" ([Improving Student Learning: A Strategic Plan for Education
Research and Its Utilization]{.underline}, 1999).

This, the first biennial report to Congress by the Director, as required
under ESRA, describes three goals that have guided the work of the
Institute and provides an overview of progress through the end of 2004.
It describes the major projects carried out within each center of the
Institute over the past two years. And, as required by ESRA, it includes
a list of all grants and contracts in excess of \$100,000 made during
the first two years of the Institute's existence.

## Major goals

### Rigor of research

Far too much education research is based on methodologies that cannot
support the questions that are addressed or the conclusions that are
drawn. For example, qualitative and interpretative methods have been
frequently deployed to address causal questions. These methods have
their place in describing program implementation and generating
hypotheses in areas in which research is just beginning, but they do not
answer the practical issues of which programs and approaches work best
for which students under what circumstances. In the quantitative realm,
correlational analyses are frequently used to advocate for or against
education policies even though every undergraduate is taught that
correlation is not causation. Statistically sophisticated models of
correlations among classes of related variables can advance our
understanding of possible relations between education processes and
outcomes, but far too much policy advocacy in education draws causal
conclusions from simple associations, e.g., "participation in the arts
enhances academic achievement," "use of the internet at home improves
science scores," and so on.

For a range of critical policy questions, the nation needs rigorous
analyses of cause and effect using the same randomized trials and
related methods that are deployed in health care and other fields where
questions of what works are of prime importance.

The Institute has moved aggressively to advance the rigor of education
research. Starting with funding announcements, proceeding to peer
review, and carrying through to funding decisions, the Institute
processes are designed to assure that successful grantees or contractors
employ rigorous methodologies.

"For a quarter of a century, American education researchers have tended
to favour qualitative and descriptive analyses over quantitative
studies.... This has now changed."

Joshua D. Angrist,

MIT

### Relevance of research

The second goal that has guided the work of the Institute is to increase
the relevance of education research. In preparation for the work of the
Institute, we conducted a purposive survey of education practitioners
and policymakers on the usefulness of research. A typical response was,
"There may be less than one percent of the existing research that's
really meaningful to teachers.... Teachers need strategies, practices.
Give them things that can help teaching and learning, things that can
help kids." While one may quibble about the percentage of research that
is relevant, virtually everyone agrees that we need much more education
research that addresses practical problems in powerful ways.

We are assuring the relevance of our research investments by tuning our
priorities to the needs of education practitioners and policymakers. For
example, as states have moved to expand their pre-K education offerings,
questions of which preschool curricula work best for whom under what
circumstances have become critically important. Responding to this need,
the Institute funded the Preschool Curriculum Evaluation Research (PCER)
Project, which involves an evaluation of 14 curricula involving over
3,800 children in 13 states. It included an evaluation of one of the
most widely used curricula among Head Start programs*.*

Later this year we will issue the first report from this project that
will identify those curricula that are effective in enhancing school
readiness.

"American education research has turned a corner."

Frederick Hess,

American Enterprise Institute

### Utilization

The third goal that has guided the Institute is to translate the results
of education research into practice. Producing new education research
that is both rigorous and relevant will help. However, the history of
other fields suggests that more is involved in the use of good research
than its mere presence. Evidence-based decisionmaking in other fields is
enhanced by decision support tools that make the results of research
available to users in easily understood forms. Education will adopt
research-based approaches much more rapidly if there are differential
consequences for decisionmakers whose choices are or are not grounded in
evidence, and if it is easy to access and use such evidence.

The What Works Clearinghouse (WWC) is critical to our effort to promote
the use of research in education decisions. It is fundamentally
different from prior national efforts to synthesize evidence on
education programs in that it relies on a set of specific and clearly
described protocols for judging the scientific quality of studies. For
the first time, an administrator faced with the choice of mathematics
curricula for middle school or tutoring methods for struggling readers,
etc., will be able to determine which programs have rigorous evidence of
effectiveness associated with them. As the results from the WWC come to
play a role in state and federal decisions on discretionary grants and
the flow of program funds to schools, we expect that practitioners will
want to consider evidence on what works, and program developers will
want to produce it.

## Scientific Peer Review Process

ESRA requires the Director of the Institute "to establish necessary
procedures for technical and scientific peer review of the activities of
the Institute" to assure that ''scientifically based research
standards'' are applied, among other things, to the funding of grant
applications. The Institute has established a new system for peer review
that is similar in many ways to the process of peer review at the
National Institutes of Health. A key provision is intended to put
distance between, on the one hand, the program officers and
administrators within the Institute who administer grant programs, work
with grantees, and disseminate the results of research, and, on the
other hand, those who are responsible for the peer review of
applications for funding under those grant programs.

"Proponents of the reorganization of research responsibilities at the
Department of Education should be at least moderately pleased."

Andrew Rotherham,

Progressive Policy Institute

To that end, an office for peer review and standards was created and
staffed within the office of

the deputy director for science of the Institute. That office selects
peer reviewers, determines review criteria, manages competitions,
provides feedback to applicants, and generates scores for applications
that determine scientific merit for funding decisions. This office also
handles peer review of all Institute reports. To support the process of
peer review of grant applications, the Institute has created an entirely
electronic application submission and review mechanism that is on par
with the best available among federal agencies.

## Human capital

More than 30 scientists have been hired to staff and provide leadership
to the Institute\'s programs. The Institute recruited an eminent
scientist, Dr. Barbara Foorman, to be commissioner for its National
Center for Education Research; a distinguished economist, Dr. Phoebe
Cottingham, to be commissioner for its National Center for Education
Evaluation and Regional Assistance; and a prominent developmental
psychologist, Dr. Lynn Okagaki, to be deputy director for science.

## The Centers

### The National Center for Education Statistics

Reliable data are critical to informed decisionmaking for improving
education in America. Indeed, the initial mission of the first
Department of Education created in 1867 was to "collect...such
statistics and facts as shall show the condition and progress of
education in the several States and territories." While today's
education and the task of assessing its scope, quality, and impact are
immensely more complex, the National Center for Education Statistics
(NCES) within the Institute of Education Sciences continues to carry out
this core function for the Department and the nation.

As reauthorized by Congress under the Education Sciences Reform Act of
2002 (Pub. L. 107-279), NCES has as its responsibility to "collect,
report, analyze, and disseminate statistical data related to education
in the United States and in other nations...." To meet this obligation,
the Center continually works with its diverse customer groups and relies
on their feedback to determine how it can best meet their demands for
timely, comprehensive, and useful information that maintains high
statistical standards. NCES strives to provide a balanced portfolio of
products and services that include data about emerging issues, while
still recognizing the need for basic statistical information about
trends in educational participation, institutions, and resources. NCES
supports a wide range of activities, carrying out a program of over 30
surveys; maintaining a user-friendly, interactive web site; and
assisting states and postsecondary institutions in building a solid
infrastructure for accurate and timely statistics through cooperative
systems with the elementary/secondary, postsecondary, and library
communities.

NCES collects statistical data on all levels of education from
preprimary through graduate study, including adult education. The
education policy issues addressed by its data collections are equally
wide ranging, including enrollment trends, access to postsecondary
education, the academic achievement of students, comparisons of the U.S.
education system with those in other countries, and the impact of
education on employment and economic productivity.

[Institutional surveys]{.underline}. NCES sponsors a core program of
administrative statistics covering institutional-level data on schools,
school districts, libraries, and postsecondary institutions. These data
are collected through state agencies and postsecondary institutions, and
form a basis for conducting sample surveys for more detailed statistics.
Elementary and secondary surveys that build on these core surveys
include the Schools and Staffing Survey and the School Survey on Crime
and Safety. The postsecondary survey program supports such surveys as
the National Postsecondary Student Aid Study and the National Study of
Postsecondary Faculty. Quick-turnaround surveys on emerging policy
concerns are conducted at both the elementary/secondary and
postsecondary levels.

[Household surveys]{.underline}. NCES conducts a National Household
Education Survey covering various education topics, such as early
childhood and adult education, program participation, education-related
home activities, and parental involvement in education. It contributes
to the Bureau of the Census population surveys for annual measures of
the dropout rate and educational attainment.

[Assessments]{.underline}. NCES conducts the National Assessment of
Educational Progress (NAEP), which assesses academic achievement at
grades 4, 8, and 12 in a number of domains, including reading,
mathematics, writing, science, U.S. history, geography, and civics.
Under the No Child Left Behind legislation, NAEP is administered every
two years in reading and mathematics, and all states are required to
participate in order to obtain Title I funds. NAEP is recognized as the
"Nation's Report Card" for providing timely, accurate, and unbiased
information on student performance nationally and in the states. NCES
also conducts a periodic nationally representative assessment of adult
literacy among the 16- to 65-year-old population.

[Longitudinal studies]{.underline}. NCES conducts a program of
longitudinal studies, which track educational developments of
individuals over time, in the context of a wide variety of educational,
social, and environmental factors. These longitudinal studies address a
variety of important education issues from early child development
through postsecondary education. Issues include factors associated with
school failure and success and the quality of education in public and
private schools.

[International assessments, surveys, and indicators]{.underline}. NCES
participates in international surveys of educational achievement and
programs to develop cross-national education data and indicators.
Working closely with such organizations as the Organization for Economic
Cooperation and Development (OECD) and the International Association for
the Evaluation of Educational Achievement (IEA), NCES plays a leading
role in the conceptualization, development, and implementation of
international surveys and programs. Among the international surveys in
which NCES participates are the Trends in International Mathematics and
Science Study (TIMSS); the Program for International Student Assessment
(PISA), a study monitoring the performance of 15-year-olds in reading,
mathematics, and science literacy; Progress in International Reading
Literacy Study (PIRLS), a comparative study of the reading literacy
skills of fourth-graders in 35 countries; and Adult Literacy and
Lifeskills (ALL) Study, an international assessment of the literacy
skills of the adult population (ages 16-65) in participating countries.

### The National Center for Education Research

The National Center for Education Research (NCER) supports rigorous
research that contributes to the solution of significant education
problems in the United States.

Through its research initiatives and the national research and
development centers, NCER engages in research activities that will
result in the provision of high quality education for all children,
improvement in student academic achievement, reduction in the
achievement gap between high-performing and low-performing students, and
increased access to and opportunity for postsecondary education.

"Over the past four years, causal questions -- questions about the
impact of alternative policies and practices -- have emerged as
priorities in educational research. These priorities are clearly
reflected in the research and evaluation agenda of the US Department of
Education."

Stephen W. Raudenbush,

University of Michigan

One of the Institute\'s major objectives has been to increase the
quality of education research. In carrying out that goal, NCER has
developed focused research competitions that target specific research
questions and research methodologies appropriate to those questions. The
number of focused competitions increased from 3 in FY 2002 to 8 in FY
2004; the number of applications received increased from 226 in FY 2002
to 600 in FY 2004, and the number of new research awards increased from
26 in FY 2002 to 46 in FY 2004.

[Performance tracking]{.underline}. We established a performance
tracking system for our research investments by submitting each year to
the same external panel of distinguished scientists a random sample of
our newly funded grant proposals for an evaluation of quality. Comparing
the base year of 2001 to 2004, there has been a 67 percent increase in
funded grants rated as high quality. Because randomized controlled
trials provide the most rigorous tests of what works in education, and
because our customers are predominantly interested in what works
questions, we have also tracked annually the proportion of our funded
proposals addressing what works questions that use experimental methods.
Again using the year before the new procedures were established as the
baseline (FY 2001), the proportion of our funded projects asking causal
questions that use randomized trials increased by over 200 percent
through FY 2004.

To increase the relevance of the Institute\'s research and evaluation
activities, we have targeted our activities to address the pressing
needs expressed by local, state, and national education decisionmakers
in our education decisionmakers survey. We have placed a much greater
emphasis on conducting research to develop, evaluate, and understand the
processes underlying curricula, instructional approaches, and education
interventions in academic content areas and in other areas relevant to
student learning and achievement. We have also addressed the need for
research to develop, evaluate, and understand what needs to be done to
increase student learning. To accomplish this objective, the Institute
established focused competitions in these areas rather than holding open
competitions in which researchers could submit applications for funding
for any topic of interest. Eight new focused research programs have been
established and are receiving ongoing funding: preschool curriculum;
teacher quality in reading; teacher quality in math and science; social
and character development; mathematics and science education; school
finance, management, and leadership; cognition and student learning; and
reading comprehension. Additionally, in the past two years, research and
development centers have been awarded in the following areas:
low-achieving schools; choice and innovation; rural schools; assessment,
standards, and accountability; and English language learners. Future
competitions will target early childhood, state and local policy, and
postsecondary education. Using 2001 as the baseline year, the proportion
of our funded projects rated as being highly relevant by a panel of
experienced state and local education leaders, including
superintendents, directors of curriculum and instruction, and chief
state school officers, increased by 138 percent through FY 2004.

[Interdisciplinary training]{.underline}. Recognizing that there are
significant capacity issues within the education research community, we
have established a program to fund interdisciplinary research training
programs in the education sciences. In FY 2004, five institutions were
awarded grants to put together a program across departments (such as
psychology, political science, economics, education, and epidemiology)
that provides intensive training in education research and statistics.
Pre-doctoral students will graduate within a traditional discipline,
e.g., economics, but will receive a certificate in education sciences,
and will be expected to produce dissertations on education topics. In FY
2004, the Institute partnered with the American Psychological
Association to establish new post-doctoral fellowships to provide
training opportunities for psychologists in education research. Based on
the success of this program, the Institute announced a new post-doctoral
training grant program open to education scientists in any discipline to
provide opportunities for training young researchers in the education
sciences.

**The National Center for Education Evaluation and Regional Assistance**

The National Center for Education Evaluation and Regional Assistance
(NCEE) is responsible for conducting rigorous evaluations of federal
programs, synthesizing and disseminating information from evaluation and
research, and providing technical assistance to improve student
achievement.

NCEE conducts---through grants, cooperative agreements or
contracts---studies that will assess the impact of education programs on
academic achievement, particularly in reading, mathematics, and science.
It reviews selected evaluation work supported by others outside the
Department, to assess the quality of studies and the significance of
such evaluations for improving education. NCEE sponsors a variety of
activities to promote the widespread adoption of rigorous evaluation
designs and methodologies in federal and non-federal education
evaluation studies.

[Rigorous evaluations.]{.underline} A major activity of the evaluation
center is identifying which federal programs are effective and which are
not. NCEE represents an important shift in the purposes of program
evaluation and the types of methodology used in evaluation studies
within the Department of Education. In FY 2003, we began a new
generation of evaluation studies designed to produce rigorous scientific
evidence of the effectiveness of programs and practices. In particular,
NCEE focused on conducting rigorous impact studies of promising
education programs and practices that are supported through federal
funds. The evaluation studies use methodologies that can provide
credible scientific evidence to answer questions of effectiveness.

Eighteen large evaluation studies were launched during 2003-2004,
including Reading First, Early Reading First, reading interventions for
struggling readers in late elementary school, education technology in
math and reading, after-school academic remediation, family literacy,
charter schools, English language acquisition, professional development,
teacher education, teacher induction, adult literacy, school choice, and
school-based violence prevention.

[Wide dissemination]{.underline}. NCEE released 10 reports in 2003-2004,
including high-visibility reports such as the *Third National Even Start
Evaluation: Follow-Up Findings From the Experimental Design Study*, and
*When Schools Stay Open Late: The National Evaluation of the 21^st^
Century Community Learning Centers Program - Second Year Findings*. In
addition, NCEE issued several guides to evidence-based practice,
including, *Identifying and Implementing Educational Practices Supported
by Rigorous Evidence: A User-Friendly Guide*.

The What Works Clearinghouse (WWC) is NCEE's pioneering effort to
synthesize the best evidence of the effectiveness of education programs,
policies, and practices. WWC reports on up-to-date research findings
available through its web site at
http://[www.whatworks.ed.gov](http://www.whatworks.ed.gov/).

All WWC reports are easily accessed, downloaded, and printed from the
WWC web site. Users can search the WWC databases for study,
intervention, and topic reports. New and updated reports are posted
online monthly. Since the WWC's interactive web site was launched in
October 2002, it has had more than 4 million visits.

"This \[the WWC\] really represents a turning point in education,
because until now, educators have had to rely on the claims of those who
produced the programs themselves or on research that may or may not have
followed rigorous design standards. For the first time, they should be
able to get the kind of information they need in choosing what to do to
make improvements in their schools and classrooms."

Karin Chenoweth,

The Washington Post

In November 2004, the WWC released the first topic level report and five
intervention reports on middle school mathematics curricula, a subject
area receiving critical attention because of its importance in state
systems measuring school performance. The central finding was that most
math curricula have not been evaluated with scientifically valid studies
that would produce scientifically valid conclusions about a curriculum's
effect on student learning.

[The Regional Educational Laboratory Program]{.underline}. Congress
created a system of Regional Labs to develop and disseminate---in
cooperation with schools, state education agencies, and research
universities---ideas and programs for improving education practices
throughout the country. The Regional Labs are in the last year of their
contracts. Contingent on continued funding from Congress, the Regional
Lab contracts will be recompeted in 2005 in the context of the same
goals that drive the Institute: rigor, relevance, and utilization.

[ERIC]{.underline}. A major accomplishment for NCEE was the March 2004
award of the new contract for the redesign of the Education Resources
Information Center (ERIC), the largest education database in the world.
The ERIC online system went live in September 2004 and now provides the
public with a centralized ERIC web site for searching the ERIC
bibliographic databases of more than 1.1 million citations going back to
1966. In October 2004, ERIC for the first time offered free public
access to more than 100,000 archived full-text documents.

[The National Library of Education]{.underline}. In 2003, the National
Center for Education Evaluation and Regional Assistance appointed a
professional librarian as Director of the National Library of Education.
The National Library of Education collects and archives information,
providing special historical and current collections of Department of
Education documents, a collection of journals supporting the ERIC
database, research reports supporting the What Works Clearinghouse, and
resources supporting current and historical federal education
legislation. The overall focus of the collection is education issues,
research, and policy. The Library also maintains a collection of
government documents, serving as a federal depository library under the
Government Printing Office.

## Grant and Contract Awards

The Institute carries out its programs through grants and contracts. The
attachment includes all awards made in the first two years of the
Institute's existence.

## Conclusion

The nation needs policymakers, educators, and concerned citizens to see
the value of rigorous evidence, to turn to it when difficult decisions
arise, and to insist that new policies that can't wait for evidence be
tested as they are implemented. That requires a transformation in the
way education is conducted. The Institute is at the center of that
transformation because that is our statutory mission and there is
substantial, bipartisan political support for evidence-based education
policy. The Institute has made substantial progress in the last two
years. Much work remains and many challenges lie ahead. Our eventual
success will depend on the continued support by Congress of the mission
and activities of the Institute.