EDUCATION 

RESEARCH

Further Improvements 

Needed to Ensure 

Relevance and Assess 

Dissemination Efforts

Report to the Committee on Education 

and the Workforce, House of 

Representatives

December 2013

GAO-14-8

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-14-8, a report to the 

Committee on Education and the Workforce, 

House of Representatives 

December 2013 

EDUCATION RESEARCH

Further Improvements Needed to Ensure Relevance

and Assess Dissemination Efforts

Why GAO Did This Study

The federal government has a 

longstanding role in conducting 

education research and collecting 

education data, and IES has a broad 

mission to provide this information to a 

wide variety of stakeholders. With a 

fiscal year 2013 budget of just under 

$600 million, IES is Education’s 

primary research and evaluation arm. 

GAO was asked to review IES’s 

performance. This report examines: (1) 

the extent to which IES has 

demonstrated its ability to support 

high-quality research and fulfill its 

mission, (2) the extent to which 

selected Education research and 

technical assistance groups 

disseminate relevant products to the 

education field, and (3) how IES 

coordinates its activities with other 

relevant federal research agencies and 

within Education. GAO reviewed 

relevant federal laws and regulations, 

agency documents and data, 

interviewed agency officials and 

stakeholders, and analyzed information 

from selected research and technical 

assistance groups. GAO also 

compared IES’s practices to federal 

internal control standards and leading 

practices for performance 

management and collaboration. 

What GAO Recommends

GAO recommends that Congress 

consider granting Education authority 

to combine funds authorized for 

evaluation of ESEA programs, and IES 

use available data to manage its peer 

review process, develop a structured

process to gather stakeholder input, 

develop performance measures that 

reflect all key agency activities, and 

assess REL and R & D Center 

dissemination strategies. Education 

agreed with our recommendations. 

What GAO Found

The Department of Education’s (Education) Institute of Education Sciences (IES) 

supports high-quality research, but lacks certain key procedures needed to fulfill 

other aspects of its mission. According to stakeholders, IES has substantially 

improved the quality of education research since its inception over a decade ago. 

However, GAO identified concerns with IES’s ability to produce timely and 

relevant research. For example, IES’s efforts are slow to respond to 

stakeholders’ needs, in part, because the time IES’s products have spent in peer 

review has substantially increased in recent years—from an average of 117 days 

in fiscal year 2011 to 150 days in fiscal year 2013—and IES does not monitor 

some aspects of these timeframes. In addition, IES does not have a structured 

process for incorporating stakeholder input into its research agenda, which 

previous GAO work has shown to be key to sound federal research programs. 

Lastly, IES’s performance measures do not fully reflect its current programs, 

which is not consistent with leading practices GAO has identified for performance 

management. For example, IES does not publicly report on the overall 

performance of the Regional Educational Laboratories (REL) program, which 

constitutes one of the agency’s largest investments. IES officials said that they 

have begun to develop new performance measures for all of their programs, but 

these will not be in place until fiscal year 2015. 

Although Education’s research and technical assistance groups have taken steps 

to disseminate relevant research to the education field, IES does not always 

assess these efforts. Some stakeholders raised concerns about the 

dissemination of relevant products from the RELs and Research and 

Development Centers (R & D Center). For example, they told GAO that these 

groups do not always adapt their products for use by both policymaker and 

practitioner audiences. Further, IES has not fully assessed REL and R & D 

Center dissemination efforts. As a result, IES does not know if these efforts are 

effective in meeting their mandated goal of providing usable research and 

information to stakeholders. GAO’s prior work on information dissemination 

suggests that further assessment could help inform IES’s oversight of the RELs 

and R & D Centers to improve these groups’ dissemination to key audiences. 

IES coordinates with other relevant federal research agencies to increase the 

use of research evidence in federal decision-making. For example, IES and the 

National Science Foundation recently released guidelines to help improve the 

quality of evidence resulting from federally-funded education research, which 

stakeholders said will benefit the education field. Within the department, IES 

plans evaluations of Education programs through coordination with various other 

offices. However, Education lacks statutory authority to combine evaluation funds 

for Elementary and Secondary Education Act (ESEA) programs, which officials 

said limits the department’s ability to conduct high-quality evaluations of 

programs it considers most important. GAO’s prior work has shown that many 

Education programs, especially smaller programs, have not been evaluated, limiting 

the ability of Congress to make informed policy decisions. For example, in 2009, 

GAO reported that 11 of Education’s teacher quality programs had not been 

evaluated in more than 7 years. Officials said that the ability to combine evaluation 

funds would allow the department to conduct needed evaluations of ESEA programs.

View GAO-14-8. For more information, contact 

George Scott at (202) 512-7215 or 

scottg@gao.gov. 
Page i GAO-14-8 Education Research 

Letter 1 

Background 3

IES Supports High-Quality Research, but Lacks Key Processes and 

Performance Measures in Some Areas 9

Research and Technical Assistance Groups Take Steps to 

Disseminate Relevant Research, but IES Has Not Fully Assessed 

These Efforts 16

IES Coordinates with Other Federal Agencies, but Fragmented 

Evaluation Funding Poses Challenges for Education’s Evaluation 

Efforts 22

Conclusions 28

Matter for Congressional Consideration 29

Recommendations for Executive Action 29

Agency Comments and Our Evaluation 30

Appendix I Objectives, Scope, and Methodology 32

Appendix II Comments from the Department of Education 39

Appendix III GAO Contact and Staff Acknowledgments 42

Related GAO Products 43

Table

Table 1: The Institute of Education Science’s Centers and Their 

Key Activities 5

Figure

Figure 1: Selected Education Research and Technical Assistance 

Groups 6

Contents 
Page ii GAO-14-8 Education Research 

Abbreviations 

AERA American Educational Research Association

Education Deparment of Education

ESEA Elementary and Secondary Education Act 

ESRA Education Sciences Reform Act of 2002 

IES Institute of Education Sciences 

NBES National Board for Education Sciences 

NICHD Eunice Kennedy Shriver National Institute of Child Health 

and Human Development 

NSF National Science Foundation 

NCEERA National Center for Education Evaluation and Regional 

Assistance 

NCER National Center for Education Research 

NCSER National Center for Special Education Research

NCES National Center for Education Statistics 

OESE Office of Elementary and Secondary Education 

OPEPD Office of Planning, Evaluation, and Policy Development 

OMB Office of Management and Budget 

REL Regional Educational Laboratory 

R & D Center Research & Development Center 

SEA state educational agency 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-14-8 Education Research 

441 G St. N.W.

Washington, DC 20548

December 5, 2013

The Honorable John Kline

Chairman

Committee on Education and the Workforce

House of Representatives

The Honorable George Miller

Ranking Member

Committee on Education and the Workforce

House of Representatives

For more than 100 years, the federal government has invested in 

conducting education research and collecting education data. More 

recently, the Education Sciences Reform Act of 2002 (ESRA) established 

the Institute of Education Sciences (IES) and outlines a broad mission for 

IES to expand fundamental knowledge and understanding of education 

and to provide this information to a wide variety of stakeholders, including 

parents, educators, researchers, policymakers, and the general public.1

As the primary research and evaluation arm of the Department of 

Education (Education), IES is charged with providing information about 

educational policies, programs, and practices that improve academic 

achievement and access to educational opportunities for all students. In 

fiscal year 2013, IES had a budget of just under $600 million, which it 

used to support a range of research, data collection, and evaluation 

activities.2

Specifically, IES funds projects on topics that range from studying the 

effectiveness of basic reading or math initiatives, to evaluating the impact 

of federal grant programs, to collecting student performance data through 

the National Assessment of Educational Progress, a nationally 

representative measure of student academic performance. You requested 

that we review IES’s performance in addressing its critical and wideranging mission. This report examines: (1) the extent to which IES has 

 

1

Pub. L. No. 107-279, Title I, Part A, § 111, 116 Stat. 1940, 1944 (codified at 20 U.S.C. 

§ 9511). 

2

According to IES, this amount does not include funds that support certain program 

evaluations conducted by IES, for example those required and funded by other statutes. 

 
 

Page 2 GAO-14-8 Education Research 

demonstrated its ability to support high-quality research and fulfill its 

mission; (2) the extent to which selected Education research and 

technical assistance groups disseminate relevant products to the 

education field; and (3) how IES coordinates its activities with other 

relevant federal research entities and within Education.

To assess the extent to which IES has demonstrated its ability to support 

high-quality research and fulfill its mission, we reviewed relevant federal 

laws and regulations as well as agency documents describing IES’s 

performance measures and key agency processes. We conducted 

interviews with officials from IES and a range of stakeholder groups 

representing researchers, policymakers, and practitioners, as well as 

several members of IES’s advisory board—the National Board for 

Education Sciences (NBES). We obtained IES data for the most recent 

four years, fiscal years 2010 through 2013, on the average number of 

work days from the submission of the initial manuscript to final approval 

for reports sponsored by IES that underwent external peer review before 

being publicly released. We assessed the reliability of IES data by 

obtaining written responses from agency officials knowledgeable about 

the data. We determined that the data were sufficiently reliable for the 

purposes of this report. To address the extent to which selected 

Education research and technical assistance groups disseminate relevant 

products, we obtained documents from each of these groups: the 

Regional Educational Laboratories (REL), the Comprehensive Technical 

Assistance Centers (Comprehensive Centers), and the Research and 

Development Centers (R & D Center). We analyzed these documents to 

identify what is known about the relevance,3 dissemination,4 and 

utilization of the research and products these groups produce. We also 

met with the directors of the RELs and Comprehensive Centers as well as 

program staff at IES and Education’s Office of Elementary and Secondary

 

3

For the purposes of this report, we define relevance as the degree to which work (1) 

provides information that could be used to guide decisions about policies, programs, or 

practices, (2) contributes new information to the research topic, and (3) builds on key 

literature or previous research. 

4

ESRA defines dissemination as “the communication and transfer of results of 

scientifically valid research, statistics, and evaluations, in forms that are understandable, 

easily accessible, and useable, or adaptable for use in, the improvement of educational 

practice by teachers, administrators, librarians, other practitioners, researchers, parents, 

policymakers, and the public, through technical assistance, publications, electronic 

transfer, and other means.” Pub. L. No. 107-279, § 102(10) (codified at 20 U.S.C. 

§ 9501(10)). For the purposes of this report, we will use this definition. 
 

Page 3 GAO-14-8 Education Research 

Education (OESE) who administer these programs. We administered a 

short survey via e-mail to the directors of IES’s R & D Centers to obtain 

additional information. We fielded this survey from April 2013 to May 2013 

and achieved a 94 percent (17 of 18) response rate. We selected these 

three research and technical assistance groups because they are all 

authorized by the same law that established IES. To describe how IES 

coordinates with other relevant federal research agencies and within the 

department, we reviewed relevant federal laws and regulations and other 

relevant documents. We also interviewed Education officials responsible 

for planning evaluations of departmental programs. Finally, we met with 

officials from two federal research organizations that conduct educationrelated research—the Eunice Kennedy Shriver National Institute of Child 

Health and Human Development (NICHD) and the National Science 

Foundation (NSF)—to learn about their processes and coordination with 

IES. For all three objectives, we compared agency documents and 

procedures to GAO’s criteria on internal controls, performance 

measurement and reporting, and collaboration, as well as a framework 

GAO developed to identify key elements of sound federal research and 

evaluation programs.5 Appendix I describes our scope and methodology 

in more detail.

We conducted this performance audit from July 2012 to December 2013 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

Prior to the establishment of IES, federal education research was carried 

out by the Office of Educational Research and Improvement, a federal 

agency that—according to observers both within and outside the federal 

 

5

GAO, Managing for Results: Executive Branch Should More Fully Implement the GPRA 

Modernization Act to Address Pressing Governance Challenges, GAO-13-518

(Washington, D.C.: June 26, 2013), Employment and Training Administration: More 

Actions Needed to Improve Transparency and Accountability of Its Research Program, 

GAO-11-285 (Washington, D.C.: Mar. 15, 2011), Results-Oriented Government: Practices 

That Can Help Enhance and Sustain Collaboration among Federal Agencies, GAO-06-15

(Washington, D.C.: Oct. 21, 2005), and Standards for Internal Control in the Federal 

Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999).

Background
 

Page 4 GAO-14-8 Education Research 

government—was challenged by frequent leadership changes, competing 

interests, and lack of a focused mission. The 2002 reauthorization of the 

Elementary and Secondary Education Act (ESEA) made numerous 

references to the use of research-based evidence in educational decision 

making and federal education programs. With its enactment later in 2002, 

ESRA established IES, an arm of the Department of Education dedicated 

to the improvement of federal education research, statistics, evaluation, 

information, and dissemination.

IES’s mission is “to provide national leadership in expanding fundamental 

knowledge and understanding of education from early childhood through 

postsecondary study, in order to provide parents, educators, students, 

researchers, policymakers, and the general public with reliable 

information about (1) the condition and progress of education in the 

United States, including early childhood education; (2) educational 

practices that support learning and improve academic achievement and 

access to educational opportunities for all students; and (3) the 

effectiveness of Federal and other education programs.” Furthermore, in 

carrying out its mission, ESRA calls upon IES to “compile statistics, 

develop products, and conduct research, evaluations, and wide 

dissemination activities in areas of demonstrated national need.”

6

ESRA authorizes IES to conduct and support many different types of 

research and evaluations in support of its mission.7 Specifically, ESRA 

contains several key provisions related to the management, core 

functions, and processes of IES:

• all research conducted by IES is to use scientifically based research 

standards that include, where appropriate, making claims of causal 

relationships only in random assignment experiments;

• education evaluations conducted by IES are to employ experimental 

designs using random assignment, when feasible;

 

6

Pub. L. No. 107-279, § 111 (b) (codified at 20 U.S.C. § 9511(b)). 

7

In this report, we may use the term IES to refer to one or more of its four Centers: the 

National Center for Education Evaluation and Regional Assistance (NCEERA), the 

National Center for Education Research (NCER), the National Center for Education 

Statistics (NCES), and the National Center for Special Education Research (NCSER). See 

table 1 for more information on these Centers and their key activities.
 

Page 5 GAO-14-8 Education Research 

• all research, statistics, and evaluation reports conducted by or 

supported through IES must be subjected to rigorous peer review 

before being published or otherwise made available to the public;

• the Director of IES will be appointed by the President, by and with 

consent of the Senate, for a 6-year term and will propose the 

Institute’s overall research priorities and report biennially to Congress 

and others on the Institute’s activities, among other duties; and

• a board of directors is established—the NBES—whose duties include, 

among other things, (1) advising and consulting with the Director of 

IES regarding its policies and approving the Director’s overall 

research priorities, (2) reviewing and approving procedures for peer 

review, and (3) reviewing the work of IES to ensure the consistency of 

scientifically valid research.

All of IES’s work is carried out by four Centers, created by law, that exist 

within IES. See table 1 for more information on these Centers and their 

key activities.

Table 1: The Institute of Education Science’s Centers and Their Key Activities

Center Key activities 

National Center for Education Evaluation 

and Regional Assistance (NCEERA)

Through contracts with private research organizations, NCEERA conducts evaluations of 

federal education programs; provides technical assistance, conducts research, and 

disseminates research findings through the Regional Educational Laboratory program; 

and disseminates information about education through the Education Resources 

Information Center and the National Library of Education. 

National Center for Education Research 

(NCER)

Through investigator-initiated research grants, NCER sponsors sustained research that 

will lead to the accumulation of knowledge and understanding of education and provide 

the basis for improving academic instruction and lifelong learning.

National Center for Education Statistics 

(NCES) 

Through contractors and its own staff, NCES collects and analyzes data, and produces 

reports containing representative education statistics about the condition of the U.S. 

education system and the academic achievement of our nation’s students over time. 

National Center for Special Education 

Research (NCSER)

Through grants to researchers, NCSER sponsors research to expand knowledge and 

understanding of the needs of infants, toddlers, and children with disabilities in order to 

improve the developmental, educational, and transitional results of such individuals. 

Source: GAO analysis of IES documents.

Education administers programs that support education research and 

technical assistance through grants and contracts involving several 

research groups—the RELs and R & D Centers within IES and the 

Comprehensive Centers within OESE (see fig. 1).
 

Page 6 GAO-14-8 Education Research 

Figure 1: Selected Education Research and Technical Assistance Groups

The National Center for Education Evaluation and Regional Assistance 

(NCEERA) administers the REL program, a network of 10 regional 

entities that conduct applied research; develop and disseminate research 

and products; and conduct technical assistance and other activities to 

support the needs of state and local educational agencies in their region. 

The RELs were first established in 1965 under Title IV of ESEA with the 

broad goal of supporting general educational improvement efforts in their 

regions. Since IES was created and began to administer the REL 
 

Page 7 GAO-14-8 Education Research 

program, the RELs have completed two contract cycles, and the current 

RELs were launched in 2012 under new 5-year contracts.

ESRA also includes specific requirements for the administration of 

National R & D Centers, which are designed to address areas of national 

need and each of which addresses at least one of the broad research 

topics outlined in the law.8 R & D Centers are also responsible for the 

production of rigorous evidence and dissemination of products that 

provide practical solutions to important education problems in the United 

States. Each R & D Center is funded for no more than 5 years, and in 

fiscal year 2013, the National Center for Education Research (NCER) and 

the National Center for Special Education Research (NCSER) 

administered 18 R & D Centers in total.

The Comprehensive Centers Program is a network of 22 technical 

assistance grantees that help to increase the capacity of state 

educational agencies (SEA) to assist districts and schools in meeting 

student achievement goals. Established in 1994, Comprehensive Centers 

provide services primarily to SEAs to enable them to assist school 

districts and schools, especially low-performing schools. Comprehensive 

Centers provide training and technical assistance in (1) the 

implementation and administration of programs authorized under ESEA 

and (2) the use of research-based information and strategies. Selected 

Comprehensive Centers focus on specific content areas and produce 

research-based information and products for use by SEAs.

In addition to IES, other entities conduct education-related research and 

evaluations, and ESRA includes general requirements for the Director of 

IES to coordinate its research and evaluation work with these entities, 

both within Education and across the rest of the federal government.9

 

8

Pub. L. No. 107-279, § 133(c)(1) (codified at 20 U.S.C. § 9533(c)(1)). R & D Centers 

were first established in the 1960s. According to statute, NCER is required to support the 

following topics of research through national R & D Centers: adult literacy; assessment, 

standards, and accountability research; early childhood development and education; 

English language learners research; improving low achieving schools; innovation in 

education reform; state and local policy; postsecondary education and training; rural 

education; teacher quality; and reading and literacy. 20 U.S.C. § 9533(c)(2). According to 

officials, NSCER’s authorizing legislation does not detail specific topics for NCSER’s R & 

D Centers, but Centers are required to contribute significantly to the solution of special 

education problems in the United States. 

9

Pub. L. No. 107-279, § 114(f)(3) (codified at 20 U.S.C. § 9514(f)(3)).
 

Page 8 GAO-14-8 Education Research 

Within Education, the Office of Planning, Evaluation, and Policy 

Development (OPEPD) conducts analyses and program evaluations on 

behalf of the department. Other federal agencies, such as NSF and 

NICHD—part of the National Institutes of Health—also support educationrelated research and the directors of these agencies serve as nonvoting 

ex officio members on the NBES.10 In addition, many nongovernmental 

organizations conduct education-related research and evaluations that 

inform general knowledge and understanding of educational policies, 

programs, and practice. For example, foundations such as the Bill & 

Melinda Gates Foundation, The William T. Grant Foundation, and The 

Spencer Foundation, conduct education research or program evaluation 

in specific topic areas to advance the public interest or their organization’s 

mission or goals. Additionally, other research organizations such as 

Mathematica Policy Research, MDRC, and SRI International, contract 

with the federal government or other clients to conduct agreed upon 

research or program evaluations on their behalf.

 

10The Director of the Census and the Commissioner of Labor Statistics are also nonvoting 

ex officio members of the NBES. 
 

Page 9 GAO-14-8 Education Research 

IES has substantially improved the education research field. In 2007, the 

Office of Management and Budget (OMB) assessed IES’s research and 

concluded that IES had transformed the quality and rigor of research 

within Education and increased the demand for scientifically based 

evidence of effectiveness in the education field as a whole.11 Likewise, 

many stakeholders told us that IES’s research standards had improved 

Education’s research and had a positive influence on education research 

generally. More specifically, several stakeholders told us that IES 

products, such as its publications of education statistics reports, were 

useful for their work. In addition, one regional comprehensive center 

director told us that a Practice Guide IES released on dropout prevention 

has become the framework that one of the states in its region is using for 

dropout prevention efforts statewide. While IES’s research grants and 

evaluations have resulted in many randomized controlled studies since 

the agency was established over 10 years ago, its research standards 

also include guidelines for the implementation of other rigorous research 

methodologies, and it has recently funded studies using those 

methodologies, such as regression discontinuity or single-case designs.12

Further, IES officials also described a variety of other types of research 

that the agency supports, such as data analyses and correlational 

analyses. IES’s support of these multiple types of methodologies allows it 

to better meet its various stakeholders’ needs.

 

11Office of Management and Budget, ExpectMore.gov: Institute of Education Sciences 

Research, accessed August 1, 2013, http://georgewbushwhitehouse.archives.gov/omb/expectmore/summary/10009008.2007.html. 

12Single-case designs are studies that involve repeated measurement of a single subject 

(e.g., a student or a classroom) and regression discontinuity designs compare outcomes 

for a treatment and control group that are formed based on the results of a preintervention 

measure. 

IES Supports HighQuality Research, but 

Lacks Key Processes 

and Performance 

Measures in Some 

Areas

IES Supports High-Quality 

Research, but Does Not 

Monitor the Timeliness of 

Some Aspects of Its Peer 

Review Process
 

Page 10 GAO-14-8 Education Research 

At the same time, IES’s research can be of limited usefulness to 

policymakers and practitioners if it is not released in a timely manner.13

Some stakeholders told us that research and evaluations supported by 

IES may not be completed soon enough to inform the decision making of 

policymakers and practitioners on important questions, which is a key 

component of its mission.14 For example, officials from one constituencybased organization for policymakers told us that IES’s evaluation of 

Education’s Race to the Top and School Improvement Grant Programs 

are of great interest to states, but by the time the results of these 

evaluations are released in 2014, states will not have much time to 

implement lessons learned from these studies before their program’s 

funding expires.15

IES’s peer review process may also exacerbate timeliness concerns. In 

order to ensure the high quality of IES’s work, ESRA requires IESsupported research reports to be peer reviewed before being published.16

 

13In 2010, we reported that some IES stakeholders said its research standards limit the 

usefulness of information for practitioners. See GAO, Department of Education: Improved 

Dissemination and Timely Product Release Would Enhance the Usefulness of the What 

Works Clearinghouse, GAO-10-644 (Washington, D.C.: July 23, 2010).

14Other factors influence IES’s ability to initiate and complete research quickly. For 

example, under the Paperwork Reduction Act, federal agencies that wish to collect data 

from the public must submit their proposed information collection instrument to OMB for 

approval. According to officials we spoke to at IES and other federal agencies, this 

process typically adds months to the timeframes for initiating a government-run study. In 

2005, we recommended that Congress consider amending the Paperwork Reduction Act 

to streamline this review process by eliminating one of its two public comment periods. 

This statutory change has not been made. See GAO, Paperwork Reduction Act: New 

Approach May Be Needed to Reduce Government Burden on Public, GAO-05-424

(Washington, D.C.: May 20, 2005).

15IES officials noted that, regardless of when funding for these specific programs ends, 

findings from these evaluations could help to inform education reform activities more 

broadly. 

16Pub. L. No. 107-279, § 186(c) (codified at 20 U.S.C. § 9576(c)). Scientific peer review is 

the responsibility of IES’s Standards and Review Office. Generally, Standards and Review 

Office staff recruit independent researchers to serve as external peer reviewers. These 

individuals (1) have published research in top research journals in the content domains 

relevant to the study presented in the report, (2) have used similar methodological 

approaches, or (3) have substantial experience conducting studies similar to the one 

presented in the report. IES products with limited data analyses or that present new 

analyses of data limited to cross-tabulations are not required to go through an external 

peer review panel before IES approves publication. However, these products are still 

reviewed internally by Standards and Review Office staff before being published. 
 

Page 11 GAO-14-8 Education Research 

In recent years however, the time it takes to complete this review process 

substantially increased, from an average of 117 days in fiscal year 2011 

to 175 days in fiscal year 2012 and 150 days in fiscal year 2013.17 As 

long ago as 2008, peer review timeliness was a concern of the NBES and 

at that time, they recommended that IES establish procedures to ensure 

the timely receipt of reports and revisions from its contractors. When 

asked for explanations for the recent increase in its peer review 

timeframes, senior IES officials told us that the timeliness of IES 

contractor responses to peer review comments may still be a factor. They 

also cited factors such as the complexity of the reports reviewed recently 

and the time it took IES’s Centers to work with its contractors on suitable 

responses to peer review comments. However, IES officials told us that 

while the peer review office within IES monitors the time its own staff take 

to review reports, the peer review office does not monitor the time IES’s 

Centers or contractors take to respond to peer review comments. Such 

monitoring would allow IES to take steps to mitigate delays, such as by 

holding contractors more accountable. In accordance with federal internal 

control standards, program managers should have access to and use 

operational data to determine whether they are meeting their agencies’ 

goals for accountability for effective and efficient use of resources.18

To ensure that its research addresses the needs of a range of 

stakeholders and to address concerns about how relevant its research is 

to these stakeholders, IES is soliciting feedback from practitioners and 

redesigning some existing programs. Current IES officials said that IES 

has in the past 10 years established the quality of IES-supported 

research, and that they are continuing to prioritize engagement with 

policymakers and practitioners. Several stakeholders with whom we 

spoke also noted that IES has recently devoted more attention to 

policymaker and practitioner outreach to improve its relevance to the 

education field. For example, IES recently convened a group of 17 state 

and local education officials and other stakeholders to discuss the 

strengths and weaknesses of products from the RELs and the What 

Works Clearinghouse, a program that IES administers to evaluate the 

 

17This time period includes time that the report is with IES’s Standards and Review Office 

for review of the original submission as well as for review of revisions and time that the 

report is with IES’s Centers or contractors for revision. 

18GAO/AIMD-00-21.3.1. 

IES Has Taken Steps to 

Meet Stakeholders’ Needs, 

but Lacks a Formal 

Process to Gather Input on 

Its Research Agenda
 

Page 12 GAO-14-8 Education Research 

merits of and disseminate education research evidence.19 According to 

IES, at this meeting participants discussed their opinions on the usability, 

relevance, and accessibility of current REL and What Works 

Clearinghouse products. In addition, in June 2013, IES officials told us 

they were planning to convene a small group of researchers to assess 

potential gaps in the research conducted by two of its Centers: NCER and 

NCSER. IES also recently held a discussion with its board of directors 

about how to make its research more relevant to policymakers and 

practitioners.

IES has also reorganized several of its existing grant programs and 

initiated a new grant program to support research that will better target 

and understand the needs of the education field. For example, one of 

these grant programs focuses on collaborations between researchers and 

state or local educational agencies, and in July 2013, six new grants were 

awarded for collaborative research projects, such as the design of a 

randomized postsecondary developmental education experiment in Texas 

and the development of a system to monitor students’ social and 

emotional learning in a school district in Nevada.

20 Additionally, IES is 

planning to fund a new R & D Center in fiscal year 2014 on knowledge 

utilization to study how education researchers can make their work more 

relevant and useful to practitioners and how practitioners can make 

productive decisions based on research evidence.

Despite IES’s recent efforts to increase the relevance of its research, IES 

does not have a structured process for incorporating feedback from 

policymakers and practitioners into its research agenda. ESRA requires 

that IES’s board of directors be composed of education researchers and 

other stakeholders, including educators, policymakers, parents, or school 

administrators.21 However, according to IES officials, IES has had 

 

19The new contract for the What Works Clearinghouse, which began in February 2013, 

emphasizes the usability and relevance of the Clearinghouse, for example, by calling for 

new types of educator-friendly products, continued improvement of the website based on 

educator feedback, and optimization of the website for tablets and smartphones.

20According to IES, 14 additional grants for supporting collaboration between researchers 

and policymakers and practitioners have been awarded since 2009 under the Evaluation 

of State and Local Education Programs and Policies grant program.

21ESRA requires that the board have 15 presidentially-appointed members, of which no 

fewer than 8 must be researchers. Pub. L. No. 107-279, § 116(c) (codified at 20 U.S.C. 

§ 9516(c)).
 

Page 13 GAO-14-8 Education Research 

difficulty with the nomination and approval process for appointing board 

members and, according to one long time board member, the NBES has, 

at times, had difficulty retaining members from these stakeholder groups. 

Further, according to IES officials, IES does not have in place any other 

ongoing, structured outreach to stakeholders for input to its research 

agenda, though the senior leaders of IES’s Centers and the Director of 

IES periodically engage with policymakers and practitioners on specific 

topics or projects. Inconsistent outreach by IES may have contributed to 

gaps in its research. For example, stakeholders said that there is a 

shortage of research using mixed methodologies that could allow for 

shorter turnaround times among IES-funded research projects.

While obtaining and integrating policymaker and practitioner feedback 

into a research agenda can be difficult, other research agencies have 

external groups of practitioner-stakeholders to which they routinely turn 

for perspectives on the overall direction of their research. Officials at 

NICHD, for example, told us that, in addition to its statutorily required 

National Advisory Council, they consult with an independent group called 

the Friends of NICHD, comprised of organizations that use NICHD 

research, to gather input on NICHD research plans and agendas. 

Similarly, in addition to the National Science Board, NSF officials told us 

that several advisory committees, composed of outside experts and 

stakeholders, regularly provide recommendations regarding the direction 

of the agency’s research. Though there is no single or ideal way for 

government agencies to conduct research, we previously developed a 

framework to identify key elements that promote a sound federal research 

program, using guidelines from several leading national research 

organizations. Within that framework, we found that agencies should 

establish a structured process for developing their research and 

evaluation priorities that considers key stakeholders’ input.22 For example, 

we reported on one federal research agency that has procedures that call 

for routinely consulting with stakeholders—including policymakers and 

 

22This framework includes five key elements: (1) agenda setting, (2) selecting research, 

(3) designing research, (4) conducting research, and (5) disseminating research results. 

See GAO-11-285. For more information on leading practices for research and evaluation 

planning, see also National Research Council, Rebuilding the Research Capacity at HUD

(Washington, D.C.: National Academies Press, 2008) and American Evaluation 

Association, An Evaluation Roadmap for a More Effective Government (September 2010), 

available at http://www.eval.org/d/do/107. 
 

Page 14 GAO-14-8 Education Research 

key institutions that influence public policy—about past research 

accomplishments and program effectiveness and impact.23

IES cannot demonstrate the impact of its efforts to improve the quality 

and relevance of its research in some areas because its performance 

measures have not been updated to reflect its current programs. Since 

1993, all federal executive branch agencies have been required to set 

strategic goals, measure performance, and report on the degree to which 

goals were met in an effort to ensure government accountability and 

enhance public awareness about agencies’ accomplishments.24 In 

addition, according to federal internal control standards and leading 

practices on performance management, agencies should establish 

performance measures for their activities and continually compare actual 

performance data against these goals.25 However, IES officials told us 

that IES’s current performance measures, which were developed after the 

agency was created by law in 2002, no longer capture the scope of IES’s 

current research and priorities. Furthermore, in some cases, senior IES 

officials told us these performance measures are no longer relevant to 

managing the agency’s operations. For example, one performance 

measure relies on the results of a survey of potential users of the What 

Works Clearinghouse, but IES officials told us they have never conducted 

this survey because they do not feel it would yield enough useful 

information to be worth the investment. To measure the effectiveness of 

its research grants, IES currently reports on a measure that counts the 

number of IES-supported interventions that have been determined to be 

effective in improving student outcomes in particular areas. While a new 

performance measure was reported in Education’s fiscal year 2014 

budget request for IES that better reflects the results of its recent 

research, this measure still does not include certain areas, such as 

 

23GAO, Employment and Training Administration: Increased Authority and Accountability 

Could Improve Research Program, GAO-10-243 (Washington, D.C.: Jan. 29, 2010).

24See, Government Performance and Results Act of 1993, Pub. L. No. 103-62, 107 Stat.

285 and GPRA Modernization Act of 2010, Pub. L. No. 111-352, 124 Stat. 3866 (2011). 

See also GAO-13-518. 

25See GAO/AIMD-00-21.3.1 and GAO, Internal Control Management and Evaluation Tool, 

GAO-01-1008G (Washington, D.C.: August 2001). For additional information about 

government performance management requirements and our related work, see our web 

page on leading practices for results-oriented management at 

www.gao.gov/key_issues/managing_for_results_in_government. 

IES’s Performance 

Measures Do Not Reflect 

Its Current Programs
 

Page 15 GAO-14-8 Education Research 

research on the organization and management of schools and education 

systems. In addition, IES does not publicly report on the overall 

performance of the REL program, which constitutes one of the agency’s 

largest investments. As we have reported, without performance measures 

that include targets or goals to demonstrate results, agencies may be at 

risk for failing to achieve their goals.26 IES officials told us they have 

begun work on revising their agency’s overall performance measures. 

Officials told us that they plan to include revised performance measures 

in Education’s fiscal year 2015 budget request for IES, and that they have 

begun discussions with OMB to establish these new measures. As of 

August 2013, IES officials told us they intend for the new performance 

measures they are developing to include all programs, including the REL 

program and its new grant programs for researcher-practitioner 

partnerships.

 

26GAO-13-518. 
 

Page 16 GAO-14-8 Education Research 

RELs, Comprehensive Centers, and R & D Centers have taken various 

steps to provide relevant research to the education field. According to 

statute, RELs and Comprehensive Centers are required disseminate 

information that can be used by practitioners and policymakers to improve 

academic achievement.27 Similarly, IES requires R & D Centers to 

disseminate research to policymakers and practitioners to improve 

teaching and learning, and ultimately, student achievement. To identify 

topics of relevance to the education field, all three groups have engaged 

policymakers and practitioners in planning research activities. For 

example, beginning in 2012, RELs were required to conduct their work 

through new or existing partnerships of practitioners, policymakers, and 

others—called research alliances, which would work together to use data 

and research to address specific problems in education. According to IES 

officials, REL projects must be based on needs identified and agreed to 

by the research alliances. For example, the REL that includes the Silicon 

Valley area works with a research alliance consisting of local school 

districts, county education officials, foundation leaders, university faculty, 

and intervention specialists to boost math achievement. This alliance 

began because local districts were concerned about low student 

achievement in math, given its importance for the local technology 

industry. With input from the alliance, this REL is developing and 

analyzing strategies to ensure adequate preparation in math with the goal 

of preparing students for postsecondary education.

 

27Pub. L. No. 107-279, §§ 174(f)(2) and 203(f)(1)(B) (codified at 20 U.S.C. §§ 9564(f)(2) 

and 9602(f)(1)(B)).

Research and 

Technical Assistance 

Groups Take Steps to 

Disseminate Relevant 

Research, but IES Has 

Not Fully Assessed 

These Efforts

Groups Have Produced 

Research and Products for 

Use by Policymakers and 

Practitioners, but 

Relevance and 

Dissemination Could Be 

Improved
 

Page 17 GAO-14-8 Education Research 

Despite efforts by these groups to ensure the relevance of their work, 

stakeholders, including teachers and policymakers, have raised concerns 

about the applicability of some of the research and products these groups 

have produced, as well as their timeliness.28 For example, a stakeholder 

group we spoke with that represents local school districts, as well as two 

superintendents we spoke with, said they did not find REL research as 

relevant or as timely as other sources they turn to for research 

information. Additionally, teachers we spoke with told us it would be 

helpful if IES-supported groups produced more products that synthesize 

research findings so that they are more applicable to classroom practice, 

similar to research-based products they use from professional 

associations or other intermediary organizations. Stakeholders also noted 

that the timeliness of REL products has been a concern. REL research 

findings cannot be released to the public until they have cleared the IES 

peer review process, which as we noted earlier, has taken longer in 

recent years.29 Some stakeholders expressed concern that in the 

absence of timely and applicable information from IES, other entities may 

provide practitioners and policymakers with research-based information 

that may not be conducted with comparable research standards or be as 

objective and unbiased as IES-supported research. In addition, the 

research topics and the products produced by the R & D Centers 

primarily reflect the priorities of researchers, according to many of the 

researchers we spoke with, even though the Centers have multiple 

audiences, including policymakers and practitioners. Stakeholder groups 

representing policymakers and practitioners also said that the R & D 

Centers could do more to adapt their research findings to formats readily 

accessible by these audiences, such as by producing nontechnical 

reports and shorter research summaries.

All three groups use a range of methods to disseminate their research 

evidence and products, such as publications and conferences. For 

 

28Comprehensive Centers conduct projects primarily for SEA clients and disseminate their 

work to these clients and to other Comprehensive Centers for their use. In 2011, IES 

issued an evaluation of the most recently completed group of Comprehensive Centers, 

including an assessment of the quality and relevance of their work for their state clients; 

therefore, we did not evaluate the relevance of the Comprehensive Centers’ products.

29IES officials told us, however, that under the most recent REL contract, interim REL 

research findings can be released to the members of the research alliances to inform 

discussion among the members. Officials said they hope this will help IES contribute to 

practice in a timelier manner, though this preliminary information is not available to those 

outside of the research alliance.
 

Page 18 GAO-14-8 Education Research 

example, in our survey of R & D Center directors, we found the common 

methods they used to disseminate their research were academic journal 

publications and presentations at conferences, as well as hosting 

conferences. Comprehensive Centers disseminate products and tools 

they produce by sharing them on their websites, via e-mail distribution 

lists, and with other Comprehensive Centers. In addition, Comprehensive 

Centers create summaries of current research for their state clients, and 

also develop other technical assistance resources specifically for 

teachers, such as professional development courses.

Although largely successful in reaching academicians, REL and R & D 

Center dissemination efforts do not always ensure that research reaches 

policymakers and practitioners. Regarding RELs, researchers and several 

groups representing policymakers and practitioners told us that REL work 

was not reaching these audiences, and teachers we spoke with were 

generally unfamiliar with RELs. A few stakeholders also noted that REL 

productivity varied widely, and that some have produced very few 

reports.30 Stakeholders gave several reasons why RELs may struggle to 

reach practitioner audiences. For example, some intermediaries—such as 

industry associations—we spoke with said their organizations help to 

disseminate research information to policymakers and practitioners. 

However, some noted that further efforts are needed to leverage 

intermediary groups to better market REL and R & D Center work to 

reach IES’s target audiences.

IES requires RELs to report some information about relevance and 

dissemination that could be used to evaluate their efforts, but this 

information has not been collected in a consistent manner and is 

therefore difficult for IES to use to improve program management. For 

example, RELs use a stakeholder feedback survey instrument to capture 

feedback from participants on relevance and other aspects of the 

products, activities, and events they sponsor. This survey includes a set 

of questions for five different types of REL activities and products, such 

as technical assistance workshops or research alliance participation. 

However, as administered in 2012—the first year of the current RELs’ 

 

30According to IES, among the 10 RELs operating during the contract period between 

2006 and 2011, 2 had not produced any impact studies after about 5 years of operation. 

However, officials noted that by the end of the contract period, all RELs had produced at 

least one impact study and many descriptive reports. 

IES Has Not Fully 

Assessed Groups’ Efforts 

to Disseminate Relevant 

Research
 

Page 19 GAO-14-8 Education Research 

operation—RELs were not consistent in how they administered the 

survey. For example, some RELs administered the complete survey for 

their activity or product type, while other RELs selected specific questions 

for respondents. The results, therefore, are not comparable across all 

REL activities and products. Further, the data collected only applies to 

participants who elected to provide feedback, rather than all participants 

in REL programs or potential users.

In addition to the feedback survey, RELs are required to annually report 

to IES on a series of indicators that they developed collaboratively with 

IES in March 2012. However, this preliminary information is not sufficient 

for IES’s management of RELs’ efforts to produce and disseminate 

relevant research. The REL program has six expected outcomes, one of 

which is to build a body of knowledge in topics that address regional 

needs. To assess performance on this and the other five program 

outcomes, RELs must report to IES on 24 indicators. For example, 

regarding dissemination, RELs must report the number of events held 

and the total number of attendees at these events. However, IES has not 

established performance targets or goals for these indicators to help 

RELs prioritize their activities or assess their performance in these areas. 

Were IES to establish targets or goals, the RELs would likely have more 

incentive to perform at the agency’s desired level, and IES would be 

better positioned to determine if the RELs are meeting its expectations 

and hold them more accountable for performance. Our previous work has 

indicated agencies successful in measuring performance had 

performance measures including targets or goals that (1) demonstrate 

results; (2) are limited to the vital few; (3) cover multiple priorities; and (4) 

provide useful information for decision making.31

Regarding the R & D Centers, IES has few requirements for tracking 

relevance and dissemination, although some Centers collect additional 

information that could be helpful for assessing performance in these 

areas. In combination with other factors, IES assesses the relevance of 

the R & D Centers’ proposed research projects when making funding 

decisions, but according to officials, this is the only formal assessment of 

the relevance of R & D Center work. However, although they are not 

required to do so, most R & D Center directors responding to our survey 

 

31GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season 

Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002). 
 

Page 20 GAO-14-8 Education Research 

said they use quantitative data to gauge their relevance to policymakers 

and practitioners. For example, one R & D Center has conducted national 

surveys of certain constituent groups, such as state policymakers, asking 

about the relevance of the Center’s findings and products for their use. In 

their annual performance reports, IES asks R & D Centers to report 

whether they disseminated research information, but does not specify 

what information they should provide, such as how they disseminated 

information or the effectiveness of these efforts. Though it can be difficult 

to demonstrate how research programs with broad dissemination goals—

such as the R & D Centers—are able to inform others and contribute to 

outcome-oriented goals, there are several evaluation strategies to assess 

how programs with great breadth and flexibility contribute to agency 

goals.32 IES has not, however, set utilization or dissemination goals and 

objectives, or investigated whether dissemination strategies were 

effective in improving the efforts of these groups.

According to officials, IES has no plans to conduct formal evaluations for 

the current group of RELs and R & D Centers to comprehensively assess 

their relevance and dissemination activities. IES is still in the process of 

conducting a mandated evaluation of the prior group of RELs, whose 

contracts ended in 2011.33 According to study design documents and IES 

officials, this IES-contracted evaluation will include information from a 

customer satisfaction survey of a representative sample of state and 

district staff regarding their views of the relevance and usefulness of REL 

work. The evaluation will also include an assessment by expert panelists 

of the quality and relevance of selected REL reports. However, it will not 

include an assessment of REL dissemination activities. This evaluation 

began in 2009 and an interim report was initially expected to be released 

in spring 2012; however, the interim report was delayed and was released 

in late September 2013.34 IES has no further plans to evaluate the REL 

program beyond the forthcoming evaluation of the prior group. Officials 

told us although there was a clear requirement in ESRA for this 

 

32GAO, Program Evaluation: Strategies for Assessing How Information Dissemination

Contributes to Agency Goals, GAO-02-923 (Washington, D.C.: Sept. 30, 2002).

33This evaluation was mandated by ESRA and was required to be carried out in the third 

year of the RELs’ contracts. See, Pub. L. No. 107-279, § 174(j) (codified at 20 U.S.C. 

§ 9564(j)).

34According to IES, the customer satisfaction data will be included in the final evaluation 

report which is estimated to be released in the spring of 2014.
 

Page 21 GAO-14-8 Education Research 

evaluation, they did not believe that the law required any subsequent 

evaluation of the REL program. In addition, as of August 2013, IES had 

no plans to conduct a formal evaluation of the R & D Center program, and 

there is no requirement to do so. According to IES officials, they do not 

plan to evaluate them because they do not view them as a program, but 

rather as one vehicle for supporting IES’s programs of research, as with 

its education research grants. However, research on dissemination 

practices funded by other federal research programs has shown that 

dissemination efforts should be assessed periodically to improve further 

efforts to communicate key messages to target audiences.35

As required by ESRA, IES has previously evaluated the Comprehensive 

Centers.36 A 2011 IES-contracted evaluation reviewed how well the 

Comprehensive Centers addressed state client needs, as well as the 

quality, relevance, and usefulness of their assistance, among other 

things.37 The evaluation included a survey of SEA administrators and a 

review of Comprehensive Center products. Evaluation data were 

collected annually in 3 of the 5 program years in which the Centers 

operated. Comprehensive Centers were rated higher in each successive 

year, and in 2008-2009, 56 percent of state managers surveyed said that 

technical assistance from the Comprehensive Centers served the state’s 

purposes completely. For the state managers who said the 

Comprehensive Centers did not completely meet their needs, the main 

reason cited was that Center staff had limited time to work with their state. 

The evaluation also included a quality assessment of a sample of 

Comprehensive Center projects by expert panelists and project 

 

35U.S. Department of Health and Human Services, Administration for Children and 

Families, Office of Planning, Research and Evaluation, The Value-Added Research 

Dissemination Framework (Washington, D.C.: February 2013). Deborah Carpenter, 

Veronica Nieva, Tarek Albaghal, and Joann Sorra, “Development of a Planning Tool to 

Guide Research Dissemination”, in Advances in Patient Safety: From Research to 

Implementation (Volume 4: Programs, Tools, and Products), ed. Kerm Henriksen et al. 

(Rockville, Md.: Agency for Healthcare Research and Quality, 2005).

36Pub. L. No. 107-279, § 204 (codified at 20 U.S.C. § 9603).

37U.S. Department of Education, Institute of Education Sciences, National Center for 

Education Evaluation and Regional Assistance, National Evaluation of the Comprehensive 

Technical Assistance Centers: Final Report (Washington, D.C.: August 2011). The 

objectives of the evaluation were to (1) describe Center operations; (2) assess the 

performance of Comprehensive Centers in addressing state needs and priorities; and (3) 

assess the quality, relevance, and usefulness of assistance provided by the 

Comprehensive Centers. 
 

Page 22 GAO-14-8 Education Research 

participants. Each sampled project was independently rated by a panel of 

experts using a 5-point scale. SEA staff who participated in these projects 

(e.g., by serving on a work group associated with the project) were asked 

via survey to rate the relevance and usefulness of each project. For the 

sample of projects assessed, the evaluation showed that the 

Comprehensive Centers’ technical assistance was rated higher on each 

measure in each successive year, from 2006-2007 to 2008-2009. On 

average, across each of the 3 years, expert panels rated sampled project 

materials from “moderate” to “high” for quality, and project participants 

rated the sampled projects “high” for relevance and usefulness. While 

these results indicate a positive trend in Comprehensive Centers’ 

performance, this evaluation covered the prior group of Comprehensive 

Centers and IES recently released a new Request for Proposals to 

evaluate the current group.

IES coordinates with relevant federal research agencies on projects to 

increase federal agencies’ use of research evidence in guiding funding 

decisions. According to federal agency officials and stakeholders we 

spoke with, NICHD and NSF are the two other primary federal agencies 

conducting education-related research. IES officials meet regularly with 

their NICHD and NSF counterparts to jointly sponsor projects and ensure 

their efforts are complementary. For example, IES co-led a joint

Education-NSF working group to develop common evidence guidelines 

IES Coordinates with 

Other Federal 

Agencies, but 

Fragmented 

Evaluation Funding 

Poses Challenges for 

Education’s 

Evaluation Efforts

IES Coordinates with 

Other Agencies to Increase 

the Use of Evidence in 

Federal Decision Making

and Research Planning
 

Page 23 GAO-14-8 Education Research 

for education research and development.

38 These guidelines are intended 

to help guide NSF’s and IES’s respective decisions about investments in 

education research and clarify for potential grantees and peer reviewers 

the evidence expected from each type of study. Several researchers we 

interviewed, as well as officials from three federal agencies we spoke 

with, said these guidelines will benefit the education field. In addition, 

developing common evidence guidelines is consistent with our key 

practices for enhancing and sustaining collaboration among federal 

agencies.39 In particular, we have found that establishing compatible 

policies, procedures, and other means helps to align the partner 

agencies’ activities, core processes, and resources to accomplish a 

common outcome.40

Consistent with this key practice, IES also coordinates with other federal 

agencies to leverage its expertise in planning federal research agendas 

on topics such as early learning and prisoner re-entry, as well as 

increasing the quality of and access to data on education outcomes. For 

example, IES sits on the Early Learning Policy Board, a joint EducationHealth and Human Services body, to discuss ways to coordinate 

federally-funded research on early childhood topics. IES’s statistics 

Center leads the Interagency Working Group on Expanded Measures of 

Enrollment and Attainment, to improve national data on education, 

training, and credentials for work. This effort includes OMB, the Council of 

Economic Advisors, and other federal statistical agencies, such as the 

Census Bureau and the Bureau of Labor Statistics at the Department of 

Labor. In addition, IES has worked with the Defense Advanced Research 

Projects Agency at the Department of Defense on its Small Business 

Innovation Research Program, and in December 2012 they released a 

joint solicitation to support the development and evaluation of education 

technology games.

 

38Institute for Education Sciences, Department of Education and National Science 

Foundation, Common Guidelines for Education Research and Development (Washington, 

D.C.: September 2013). 

39In this report we use the term “coordination” broadly to include interagency activities that 

others have variously defined as “collaboration,” “cooperation,” “integration,” or 

“networking.” 

40GAO-06-15. 
 

Page 24 GAO-14-8 Education Research 

IES’s efforts to help develop a systematic, transparent evidence review 

process and incorporate a tiered evidence framework for awarding grants 

for Education’s Investing in Innovation Fund grant program have had an 

impact on the use of evidence in federal decision making both within and 

outside the department. According to an official from the Administration of 

Children and Families at the Department of Health and Human Services, 

for some of its grant programs, the Administration of Children and 

Families has recently used tiered evidence standards similar to those IES 

helped to develop for Education’s Investing in Innovation Fund, reserving 

the majority of funds for applicants proposing to implement service 

models with evidence of effectiveness. IES is also working with Education 

program offices to incorporate tiered evidence standards in awarding their 

grants, such as the Strengthening Institutions Program in the Office of 

Postsecondary Education.

In addition to working with other federal agencies, IES coordinates within 

Education to facilitate coordination between the REL and Comprehensive 

Center programs. There is a statutory requirement for each 

Comprehensive Center to coordinate its activities and collaborate with the 

RELs in its region, as well as with certain other entities.41 Prior to the 

current award cycle, IES and OESE restructured REL and 

Comprehensive Center regions so that the number of Comprehensive 

Centers in each REL region was reduced and aligned more closely with 

REL regions. According to OESE officials, its Comprehensive Center 

grantees and states found the lack of alignment between the regions in 

the prior contract cycle to be a barrier to coordination. Further, IES has 

provided guidance regarding its expectations for coordination between 

RELs and Comprehensive Centers. For example, IES has encouraged 

RELs to partner with Comprehensive Centers and has allowed RELs to 

budget resources for these collaborative efforts.

According to directors of these groups, coordination has improved with 

the current group of RELs and Comprehensive Centers, which began in 

2012.42 One REL Director we spoke with told us that the realignment of 

the regions has facilitated improved coordination between the groups. In 

 

4120 U.S.C. § 9602(f)(2).

42The current RELs began operating in January 2012, while the current Comprehensive 

Centers began in fall 2012.

Coordination between the 

RELs and Comprehensive 

Centers Has Improved
 

Page 25 GAO-14-8 Education Research 

addition, some directors said that the RELs’ new structure of research 

alliances has improved coordination. For example, some Comprehensive 

Centers are members of REL research alliances and have partnered with 

RELs to host “bridge” events, where state officials and researchers meet 

to discuss research findings relevant to issues of policy and practice.

REL directors told us that although there can still be confusion among 

some SEA officials about the appropriate role and tasks performed by 

each, RELs and Comprehensive Centers have used several strategies to 

reduce confusion and improve coordination. For example, some groups 

have conducted joint visits with SEA officials so that they can describe 

their respective roles and responsibilities and specifically discuss how 

each group can address the state’s various technical assistance needs. In 

addition, some directors described their groups’ efforts to sequence their 

work for their state clients to better meet their needs. In some instances, 

for example, the Comprehensive Center may conduct initial planning work 

for a project, and the REL would later assist with any aspects of the 

project requiring data analysis or original research. For example, one 

Comprehensive Center director told us that, through joint efforts with the 

REL in their region, it developed a framework for measuring teacher 

effectiveness. For this project, the REL formed a research alliance with 

other Comprehensive Centers and SEAs interested in piloting the 

framework. The Comprehensive Center is helping states implement the 

pilot, and the REL will conduct an evaluation of the pilot projects. This 

Center Director told us that it will help the states adjust their strategies 

based on the findings from the pilot evaluation.

IES coordinates within the department to plan evaluations of Education 

programs. Beginning in 2010, OPEPD and IES have jointly led an annual 

department-wide evaluation planning process to identify evaluation 

projects to conduct.43 Although there is no official guidance delineating 

these roles, Education officials said that the annual evaluation planning 

process is used to help determine which of these offices will conduct each 

project. According to Education, this planning process has helped to 

 

43See GAO, Program Evaluation: Experienced Agencies Follow a Similar Model for 

Prioritizing Research, GAO-11-176 (Washington, D.C.: Jan. 14, 2011). Specifically, the 

Policy and Programs Studies Service within OPEPD is responsible for program and policy 

evaluation. Within IES, NCEERA is primarily responsible for conducting evaluations and 

coordinating with OPEPD. 

Education’s Inability to 

Combine Evaluation 

Funding Poses Challenges 

for Evaluation Planning 

Efforts
 

Page 26 GAO-14-8 Education Research 

improve the quality of evaluations conducted, and made evaluations more 

responsive to the information needs of the department. For example, IES 

is conducting an implementation and quasi-experimental evaluation of the 

Race to the Top Program. An IES official reported that input from the 

Education program office running this program helped ensure that the 

evaluation’s data collection instruments focused on implementation 

issues most important to Education and state education leaders. IES is 

also working with this program office to select topics of greatest 

immediate interest to Education for short implementation briefs to be 

released as the study progresses.

However, efforts to coordinate internally and prioritize evaluation research 

projects through this annual process are challenged in part by statutory 

requirements related to ESEA evaluation funding. For these programs, 

evaluation funds are typically set aside as a percentage of program 

funding or as national activities funds.44 With some exceptions, ESEA 

authorizes Education to reserve up to 0.5 percent of the amount 

appropriated to carry out each program or project authorized under that 

Act to conduct evaluations.45 However, according to Education officials, 

due to the statutory limitation, the funds available for evaluation are often 

insufficient to conduct high-quality evaluations of these programs.46 This 

is particularly true with respect to certain discretionary grant programs 

where overall program funding is itself relatively small. As a result, 

OPEPD and IES officials said that some evaluations, including highpriority evaluations, may not occur. For example, IES officials told us that 

the department is interested in conducting evaluations of the Mathematics 

and Science Partnerships and Promise Neighborhoods programs, but 

there is not sufficient evaluation funding under these programs due to the 

 

44According to Education, national activities funds are program-specific, but multipurpose; 

they can be used on a variety of activities, including evaluations or technical assistance, 

but only for the designated program. 

45Education may reserve up to 0.5 percent of program appropriations to conduct 

comprehensive program evaluations, evaluate the effects and cost efficiencies across 

federal education programs, and increase the usefulness of evaluations of grant 

recipients. 20 U.S.C. § 7941(a). ESEA also provides that where funds are authorized to be 

used for program evaluation under a specific provision, Education may not use the 

general authority described above to reserve additional funds for evaluation of that 

program. 20 U.S.C. § 7941(c).

46Where the amount available for evaluation is insufficient, Education officials told us that 

the funds are redistributed to program implementation activities.
 

Page 27 GAO-14-8 Education Research 

statutory limitation and Education does not have the authority to combine 

evaluation funds across the department and use them to conduct 

evaluations of these or other high-priority programs.47

We have previously reported that many Education programs, especially 

smaller programs, have not been evaluated, which can limit the ability of 

Congress to make informed decisions about which programs to continue, 

expand, modify, consolidate, or eliminate.48 For example, in 2009, we 

reported that 11 of Education’s programs focused on teacher quality had 

been operating for over seven years and had never been evaluated.49 In 

addition, according to OPEPD officials, ideas for some high-priority 

evaluations that department officials have discussed during the annual 

evaluation planning process are never developed into plans because 

officials know that evaluation funds are not available to carry them out. 

However, according to officials from OPEPD, beginning in fiscal year 

2015, Education will maintain a list of all of the department’s evaluation

needs should funds for conducting these evaluations become available. 

According to a senior IES official, if the department were able to combine 

funds authorized to be used for evaluation it would have more flexibility to 

conduct appropriate evaluations of any of ESEA programs that need 

evaluation. The President’s Fiscal Year 2014 Budget request for 

Education contained a proposal to increase Education’s flexibility to 

conduct program evaluations by allowing the department to use funds 

from certain programs across the department for this purpose.50 An OMB 

staff member told us that the ability to combine evaluation funds and 

prioritize evaluations is critical in the current environment of constrained 

funding to help make the best use of limited resources and to fill gaps in 

 

47In addition to the limitation contained in ESEA, under a generally applicable provision of 

federal law, appropriations may only be applied to the purposes for which the 

appropriations were made, unless otherwise authorized. 31 U.S.C. § 1301(a). 

48GAO, Opportunities to Reduce Potential Duplication in Federal Teacher Quality 

Programs, GAO-11-510T (Washington, D.C.: Apr. 13, 2011).

49GAO, Teacher Quality: Sustained Coordination among Key Federal Programs Could 

Enhance State Efforts to Improve Teacher Quality, GAO-09-593 (Washington, D.C.: July 

6, 2009). 

50Specifically, the proposal would authorize the department to reserve not more than 0.5 

percent from each appropriation for specified accounts in order to carry out evaluations of 

any of the programs or activities that are funded under such accounts. Office of 

Management and Budget, Budget of the U.S Government, Fiscal Year 2014, (Washington, 

D.C.: Apr. 10, 2013).
 

Page 28 GAO-14-8 Education Research 

topic areas where research is needed. In fiscal year 2012, Congress 

granted the Department of Labor temporary authority to combine 

program-specific funds to support evaluations of its programs.51 The 

Department of Labor is required to provide a plan to the congressional 

appropriations committees before combining its funds in this manner. This 

plan is to describe the specific evaluations it plans to carry out using 

these funds and helps ensure transparency and accountability in the 

department’s use of this authority. In the Department of Labor’s fiscal 

year 2014 budget justification, it reported that the availability of flexible 

evaluation funds has allowed the department to evaluate large and highpriority programs, as well as programs that may not have been fully 

evaluated previously.

Since its creation more than a decade ago, IES has made significant 

contributions to strengthening the rigor of the education research field and 

has promoted the use of scientifically based research in our nation’s 

education system. However, IES’s ability to release timely information to 

policymakers and practitioners is exacerbated by its peer review process 

which, while promoting rigorous research, can take a substantial amount 

of time. Without better use of available data to manage this peer review 

process, it will remain difficult for IES officials to identify potential causes 

of these delays and develop strategies to mitigate these challenges. More 

recently, the agency has begun working to make its products more 

relevant to a range of stakeholders, particularly policymakers and 

practitioners. Producing education research that is relevant for these 

stakeholders is critical to inform decision making in Congress and in the 

field about key federal investments in education. Despite progress IES 

has made in ensuring that its research is relevant to its stakeholders, IES 

does not have a formal, structured process for engaging with these 

groups in the research-agenda setting process. Until such a process is in 

place, IES limits its ability to ensure the relevance, and ultimately, 

usefulness of its research to the field. IES’s relevance to the field is 

unclear in other ways as well. For example, without publicly reported 

performance measures for key investments—such as the RELs or new 

 

51The Department of Labor Appropriations Act of 2012 allowed Labor to use up to 0.5 

percent of funds appropriated under certain major program accounts in order to carry out 

evaluations through fiscal year 2013. Further, the department was permitted to transfer 

these funds to its Office of the Chief Evaluation Officer. See Pub. L. No. 112-74, 125 Stat. 

786, 1063-64.

Conclusions
 

Page 29 GAO-14-8 Education Research 

high-priority grant programs—stakeholders, such as Congress and the 

public, are unable to determine the effectiveness of IES’s activities or 

evaluate the merits of the federal investment in these programs.

IES’s research and technical assistance grantees and contractors take a 

number of steps to help ensure they produce and disseminate relevant 

research and products. However, without assessing the effectiveness of 

these groups’ dissemination efforts, IES may miss opportunities to think 

more strategically about dissemination and ensure that these groups are 

using the most effective strategies to reach their target audiences. 

Providing Education with the authority to combine funds authorized for 

evaluations under ESEA would allow the department more flexibility to 

prioritize and conduct effective evaluations of programs that represent 

key federal investments and critical subject areas. Without this authority, 

it will be difficult for Education to make the best use of its limited 

resources and to support Congress in making informed decisions about 

which programs to continue, expand, modify, consolidate, or eliminate.

To ensure that Education can conduct critical program evaluation work, 

Congress should consider granting Education authority to combine funds 

authorized for evaluations of ESEA programs and target them to highpriority evaluations, with appropriate measures to ensure transparency 

and accountability for how the funds will be used.

To improve the management and accountability of IES’s research and 

evaluation efforts, we are making the following recommendations:

1. IES should use available data to routinely monitor all stages of its 

peer review process and identify opportunities to improve the 

timeliness of its reviews.

2. IES should develop a structured process to systematically gather 

input from policymakers and practitioners and use this input when 

developing its research agenda.

3. IES should develop performance measures, including targets and 

goals that clearly demonstrate results, and that reflect its current 

programs and all key agency activities, such as the performance of 

the RELs and its new grant programs supporting researcherpractitioner partnerships.

4. In order to identify leading practices and target areas for 

improvement, IES should assess the effectiveness of REL and R & D 

Matter for 

Congressional 

Consideration

Recommendations for 

Executive Action
 

Page 30 GAO-14-8 Education Research 

Center dissemination strategies by, for example, collecting consistent 

data and lessons learned from these groups to inform future 

dissemination efforts.

We provided a draft of this report to the Department of Education for 

review and comment. Their comments are reproduced in appendix II.

Education officials also provided technical comments, which we 

incorporated into the final report, as appropriate.

Education generally agreed with our recommendations and has several 

plans in place to address them. Specifically, IES noted that it continues to 

be committed to producing high-quality reports while shortening the time 

period for its peer review process, and stated that it will develop a system 

to more comprehensively monitor all of the stages in its report production 

process so that it is able to identify and address potential problems more 

easily and quickly. To further emphasize its commitment to obtaining 

practitioner and policymaker input, IES also said it will begin an annual 

solicitation for practitioner and policymaker input on its research agenda 

by posting an invitation on its web site and widely advertising the 

opportunity through its IES Newsflash email notices.

Additionally, as we noted in our report, IES commented that it is currently 

working to align its performance measures with its current investments, 

including developing measures for the REL program. Finally, IES agreed 

that it is important to measure and assess the effectiveness of IESsupported dissemination activities. IES noted it will build knowledge in this 

area through its fiscal year 2014 request for applications for a new R & D 

Center focused on knowledge utilization to develop measures for how 

practitioners use research evidence to guide what they do in schools and 

classrooms. We support IES’s continued focus and emphasis on 

advancing knowledge and providing usable information to improve 

education policy and practice.

We are sending copies of this report to relevant congressional 

committees and the Secretary of Education. In addition, the report will be 

available at no charge on GAO’s website at http://www.gao.gov. 

Agency Comments 

and Our Evaluation
 

Page 31 GAO-14-8 Education Research 

If you or our staffs have any questions about this report, please contact 

me at (202) 512-7215 or scottg@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. GAO staff who made major contributions to this report 

are listed in appendix III.

George A. Scott

Director, Education, Workforce,

and Income Security Issues 
Appendix I: Objectives, Scope, and 

Methodology

Page 32 GAO-14-8 Education Research 

Our review examined (1) the extent to which the Institute for Education 

Sciences (IES) demonstrates its ability to support high-quality research 

and fulfill its mission; (2) the extent to which selected Department of 

Education (Education) research and technical assistance groups 

disseminate relevant products to the field; and (3) how IES coordinates its 

activities with other relevant federal research entities and within 

Education. For all three objectives, we reviewed the Education Sciences 

Reform Act of 2002 (ESRA) to identify statutory requirements related to 

IES management, processes, and procedures. We obtained and 

analyzed agency documents detailing IES’s processes and performance. 

We also conducted interviews with senior IES officials, including 

Commissioners at all four IES Centers1 as well as selected program staff, 

several current and former members of IES’s advisory board, the National 

Board for Education Sciences, and selected former senior IES officials to 

gather additional perspectives on IES’s management and performance 

over time.

Specifically, to assess the extent to which IES has demonstrated its ability 

to support high-quality research and fulfill its mission, we reviewed 

documents and data from IES’s Standards and Review Office, which 

administers the peer review of IES reports and grant applications. In part 

to evaluate IES’s timeliness in producing relevant research, we obtained 

IES data for the most recent 4 years, fiscal years 2010 through 2013, on 

the average number of work days from the submission of the initial 

manuscript to final approval for reports sponsored by IES that underwent 

external peer review before being publicly released. We assessed the 

reliability of IES data by obtaining written responses from agency officials 

knowledgeable about the data. We determined that the data were 

sufficiently reliable for the purposes of this report. To further assess IES’s 

overall processes and operations, we applied a framework GAO 

previously developed to evaluate the soundness of federal research and 

evaluation programs. This framework includes five key phases of the 

research and evaluation process: (1) agenda setting, (2) selecting 

research, (3) designing research, (4) conducting research, and (5) 

 

1

IES’s four Centers are: (1) the National Center for Education Evaluation and Regional 

Assistance; (2) the National Center for Education Research; (3) the National Center for 

Education Statistics; and (4) the National Center for Special Education Research.

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology

Page 33 GAO-14-8 Education Research 

disseminating research results.2 While we assessed IES’s activities in 

each of the five phases, we primarily focused on the first and fifth phases 

of this process: agenda setting and disseminating research, as these two 

phases most directly addressed our research questions.

Additionally, we reviewed publicly reported performance information 

contained in Education’s most recent budget request to Congress for IES 

to compare performance trends over time and to assess the 

completeness of IES’s performance measures against GAO’s leading 

practices for performance management and federal standards for internal 

control in the areas of information and communications, and performance 

reporting and monitoring.3 We also interviewed officials at the research 

and evaluation arm of the Administration for Children and Families at the 

Department of Health and Human Services, and at the Office of 

Management and Budget that were recommended to us as 

knowledgeable of federal research and performance reporting to obtain 

their insight on IES’s research, evaluation, and performance 

measurement. Lastly, we observed two public meetings of the National 

Board for Education Sciences that took place during the course of our 

review on October 5, 2012, and February 22, 2013.

We also conducted more than 40 interviews with several categories of 

IES stakeholders: (1) education researchers; (2) organizations 

representing education policymakers and state-level administrators; and 

 

2

GAO, Employment and Training Administration: More Actions Needed to Improve 

Transparency and Accountability of Its Research Program, GAO-11-285 (Washington, 

D.C.: Mar. 15, 2011). 

3

GAO, Managing for Results: Executive Branch Should More Fully Implement the GPRA 

Modernization Act to Address Pressing Governance Challenges, GAO-13-518

(Washington, D.C.: June 26, 2013), Internal Control Management and Evaluation Tool, 

GAO-01-1008G (Washington, D.C.: August 2001), and Standards for Internal Control in 

the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). For 

additional information about government performance management requirements and our 

related work, see our web page on leading practices for results-oriented management at 

www.gao.gov/key_issues/managing_for_results_in_government. 
Appendix I: Objectives, Scope, and 

Methodology

Page 34 GAO-14-8 Education Research 

(3) practitioner organizations, individual teachers, and district leaders.4 To 

identify appropriate stakeholders in all three categories, we obtained 

recommendations on individuals or groups to contact from key 

organizations, contacted those individuals or groups, and obtained further 

recommendations from them on additional individuals or groups to 

contact.5 We continued to ask for names from the subsequent individuals 

and organizations until we began getting duplicate referrals. We also 

identified individuals and groups to contact through our prior work on 

education research and internal expertise. For the education researchers, 

we sought to identify education researchers that were considered to be 

knowledgeable about education research and the activities of IES and its 

predecessor organizations.6 For the education policymaker organizations, 

we ensured representation from groups that represent the wide range of 

topical areas in which IES funds research and evaluation, such as early 

childhood education, special education, and postsecondary education. 

We interviewed teachers and superintendents because they are part of 

IES’s target population of end-users, though they had various levels of 

familiarity with IES prior to our interview.

We conducted two discussion groups with experienced education 

researchers at the 2013 Annual Conference of the American Educational 

Research Association (AERA). We assembled senior researchers for 

these discussion groups with the help of AERA staff. Invitations to 

participate in these discussion groups were provided to selected 

members of the AERA Fellows program that were culled by AERA staff to 

 

4

We met with a total of 22 researchers (including individual researchers and research 

organizations), 7 policymaker organizations, 6 national organizations representing 

practitioners, and 11 individual teachers and superintendents (some teachers and 

superintendents were not available to interview but provided written responses to our 

questions) for a total of 46 interviews. In some cases, stakeholders could fall into multiple 

categories, but we have only counted each once. Stakeholder interviews generally 

covered topics related to both our first and second research objectives, though the 

emphasis differed slightly based on the subject’s realm of knowledge.

5

For the interviews with teachers, we were assisted by the National Council of Teachers of 

English and the National Council of Teachers of Mathematics who contacted individual 

teachers on our behalf. To identify superintendents for interviews, we were assisted by the 

American Association of School Administrators. 

6

In some cases, we interviewed research organizations that may have been current or 

former IES grantees or contractors. In these instances, we asked these organizations to 

disclose their current or former IES grants or contracts and sought their input because of 

their expertise and substantial knowledge of the field. 
Appendix I: Objectives, Scope, and 

Methodology

Page 35 GAO-14-8 Education Research 

ensure representation from a balanced range of subject matter areas.7

During these discussion groups, we met with a total of 16 senior 

education researchers and discussed issues such as IES’s research 

standards, peer review process, and the relevance and dissemination 

practices of IES overall, as well as those of selected IES research and 

technical assistance groups, specifically—the Regional Educational 

Laboratories (REL) and Research & Development Centers (R & D 

Center).

To address our second research objective, we focused on three types of 

research and technical assistance groups. Two are administered by 

IES—the RELs and R & D Centers—and one is administered by the 

Office of Elementary and Secondary Education (OESE)—the 

Comprehensive Technical Assistance Centers (Comprehensive Centers). 

We selected these three research and technical assistance groups 

because they are all authorized by the same law that established IES. 

Moreover, there is a statutory requirement for the Comprehensive 

Centers to coordinate their activities and to collaborate with the RELs, 

among others. Specifically, to assess the extent to which these groups 

disseminate relevant products to the education field, we obtained and 

analyzed the most recent annual performance report submitted to IES 

and OESE, respectively, from each of these groups. We analyzed these 

documents to identify what quantitative and qualitative information IES 

collects about the relevance, dissemination, and utilization of the research 

and products these groups produce. We also interviewed Directors of the 

RELs and the Comprehensive Centers as well as program staff at IES 

and OESE that administer the programs to discuss relevance, 

dissemination, and coordination activities. To further understand the 

Comprehensive Centers activities related to the production and 

dissemination of relevant research and products to the field, we reviewed 

a contractor-led evaluation of the Comprehensive Center Program. In 

particular, we reviewed findings related to relevance of the products these 

groups produced and the levels of satisfaction reported by their 

customers. We also reviewed early study design documents and an 

 

7

The AERA Fellows program honors individuals with substantial research 

accomplishments and exceptional contributions to education research. Fellows are 

nominated by their peers, selected and recommended by the Fellows Committee, and 

approved by the AERA Council, the Association’s elected governing group. 
Appendix I: Objectives, Scope, and 

Methodology

Page 36 GAO-14-8 Education Research 

interim report for an ongoing evaluation of the REL program.8 We 

reviewed the scope and methodology for each of these evaluations and 

determined they were sufficiently reliable for our reporting purposes. We 

also obtained information from IES staff responsible for overseeing these 

evaluations to obtain additional information.

We administered a short survey to the Directors of IES’s 18 R & D 

Centers that were operational in the most recent full program year (2011-

2012) at the time of our review. We asked questions about activities the 

R & D Centers undertake to disseminate relevant research to the field 

and what qualitative and quantitative data they collect on their activities. 

We conducted pretests to check that (1) the questions were clear and 

unambiguous, (2) terminology was used correctly, (3) the survey did not 

place an undue burden on respondents, (4) the information could be 

feasibly obtained, and (5) the survey was comprehensive and unbiased. 

We conducted two pretests with R & D Centers, one grantee of the 

National Center for Education Research and one grantee of the National 

Center for Special Education Research. We made changes to the survey 

after both pretests based on the feedback we received. We sent the 

survey by e-mail in an attached Microsoft Word form on April 2, 2013. 

Surveys were completed by 17 of 18 R & D Center Directors, for a 94 

percent response rate, by May 10, 2013. Because this was not a sample 

survey, it has no sampling errors. However, the practical difficulties of 

conducting any survey may introduce errors, commonly referred to as 

nonsampling errors. For example, difficulties in interpreting a particular 

question, sources of information available to respondents, or entering 

data into a database or analyzing them can introduce unwanted variability 

into the survey results. We took steps in developing the questionnaire, 

collecting the data, and analyzing them to minimize such nonsampling 

error. 

To describe how IES coordinates with other relevant federal research 

agencies and within the department, we reviewed pertinent documents 

and statutory requirements for coordination contained in ESRA and 

 

8

U.S. Department of Education, Institute of Education Sciences, National Center for 

Education Evaluation and Regional Assistance, National Evaluation of the Comprehensive 

Technical Assistance Centers: Final Report (Washington, D.C.: August 2011) and U.S. 

Department of Education, Institute of Education Sciences, National Center for Education 

Evaluation and Regional Assistance, Evaluation of the Regional Educational Laboratories: 

Interim Report (Washington, D.C.: September 2013).
Appendix I: Objectives, Scope, and 

Methodology

Page 37 GAO-14-8 Education Research 

compared agency efforts to criteria we previously developed for practices 

agencies can use to help enhance and sustain interagency collaboration.9

To identify relevant federal research agencies, we obtained 

recommendations on federal research agencies that conduct educationrelated research during interviews with knowledgeable stakeholders. We 

continued to ask for recommendations during stakeholder interviews until 

we established that the vast majority of recommendations identified two 

other federal research agencies: the education directorate at the National 

Science Foundation (NSF) and the Eunice Kennedy Shriver National 

Institute of Child Health and Human Development (NICHD), an arm of the 

National Institutes of Health. While stakeholders and officials at IES, NSF, 

and NICHD noted some key differences between these agencies, such as 

the types of research funded,10 there was general consensus that there 

were clear commonalities in the research missions and some of the topics 

of research funded by all three agencies. As such, we interviewed officials 

at NSF and NICHD about their processes and procedures as well as their 

coordination with IES, asking for examples of coordination activities and 

any potential areas for improvement. Additionally, during our interviews 

with IES officials, we asked about their coordination activities with NSF 

and NICHD. To describe IES’s coordination within the department, we 

reviewed agency documents such as those describing the annual 

evaluation planning process at Education and conducted interviews with 

IES officials and officials responsible for evaluation and research within 

the department’s Office of Program Evaluation and Policy Development. 

Finally, we also asked about coordination activities between the REL and 

Comprehensive Centers programs during our interviews with program 

staff administering these programs, as well as during our interviews with 

Directors of these groups, to assess the extent and nature of the 

coordination between these technical assistance programs.

We conducted this performance audit from July 2012 to December 2013 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

 

9

GAO, Results-Oriented Government: Practices that Can Help Enhance and Sustain 

Collaboration among Federal Agencies, GAO-06-15 (Washington, D.C.: Oct. 21, 2005). 

10According to officials, IES typically funds applied science research while NSF and 

NICHD typically fund basic science research. 
Appendix I: Objectives, Scope, and 

Methodology

Page 38 GAO-14-8 Education Research 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.
Appendix II: Comments from the Department 

of Education

Page 39 GAO-14-8 Education Research 

Appendix II: Comments from the Department 

of Education
Appendix II: Comments from the Department 

of Education

Page 40 GAO-14-8 Education Research 
Appendix II: Comments from the Department 

of Education

Page 41 GAO-14-8 Education Research 
Appendix III: GAO Contact and Staff 

Acknowledgments

Page 42 GAO-14-8 Education Research 

George A. Scott, (202) 512-7215 or scottg@gao.gov

In addition to the contact named above, Scott Spicer, Assistant Director; 

Lucas Alvarez, Nora Boretti, and Dana Hopings made significant 

contributions to this report. Also contributing to this report were Sandra 

Baxter, Deborah Bland, David Chrisinger, Elizabeth Curda, Helen 

Desaulniers, Sheila McCoy, Jean McSween, Mimi Nguyen, James 

Rebbe, and Sarah Veale.

Appendix III: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments
Related GAO Products

Page 43 GAO-14-8 Education Research 

Education Research: Preliminary Observations on the Institute of 

Education Sciences’ Research and Evaluation Efforts. GAO-13-852T. 

Washington, D.C.: September 10, 2013.

Managing for Results: Executive Branch Should More Fully Implement 

the GPRA Modernization Act to Address Pressing Government 

Challenges. GAO-13-518. Washington, D.C.: June 26, 2013.

Employment and Training Administration: More Actions Needed to 

Improve Transparency and Accountability of Its Research Program. 

GAO-11-285. Washington, D.C.: March 15, 2011.

Program Evaluation: Experienced Agencies Follow a Similar Model for 

Prioritizing Research. GAO-11-176. Washington, D.C.: January 14, 2011.

Department of Education: Improved Dissemination and Timely Product 

Release Would Enhance the Usefulness of the What Works 

Clearinghouse. GAO-10-644. Washington, D.C.: July 23, 2010.

Employment and Training Administration: Increased Authority and 

Accountability Could Improve Research Program. GAO-10-243. 

Washington, D.C.: January 29, 2010.

Program Evaluation: Strategies for Assessing How Information 

Dissemination Contributes to Agency Goals. GAO-02-923. Washington, 

D.C.: September 30, 2002.

Education Research: Education Should Improve Assessments of R&D 

Centers, Regional Labs, and Comprehensive Centers. GAO-02-190. 

Washington, D.C.: January 24, 2002.

Related GAO Products

(131176)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.