```

[House Hearing, 113 Congress]
[From the U.S. Government Publishing Office]

                     EDUCATION RESEARCH: EXPLORING
                    OPPORTUNITIES TO STRENGTHEN THE
                    INSTITUTE OF EDUCATION SCIENCES

=======================================================================

                                HEARING

                               before the

                         COMMITTEE ON EDUCATION
                           AND THE WORKFORCE
                     U.S. HOUSE OF REPRESENTATIVES

                    ONE HUNDRED THIRTEENTH CONGRESS

                             FIRST SESSION

                               __________

           HEARING HELD IN WASHINGTON, DC, SEPTEMBER 10, 2013

                               __________

                           Serial No. 113-30

                               __________

  Printed for the use of the Committee on Education and the Workforce

[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]

                   Available via the World Wide Web:
                       www.gpo.gov/fdsys/browse/
           committee.action?chamber=house&committee=education
                                   or
            Committee address: http://edworkforce.house.gov

                                _____

                  U.S. GOVERNMENT PRINTING OFFICE

82-662 PDF                WASHINGTON : 2013
-----------------------------------------------------------------------
For sale by the Superintendent of Documents, U.S. Government Printing 
Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; DC 
area (202) 512-1800 Fax: (202) 512-2104  Mail: Stop IDCC, Washington, DC 
20402-0001

                COMMITTEE ON EDUCATION AND THE WORKFORCE

                    JOHN KLINE, Minnesota, Chairman

Thomas E. Petri, Wisconsin           George Miller, California,
Howard P. ``Buck'' McKeon,             Senior Democratic Member
    California                       Robert E. Andrews, New Jersey
Joe Wilson, South Carolina           Robert C. ``Bobby'' Scott, 
Virginia Foxx, North Carolina            Virginia
Tom Price, Georgia                   Ruben Hinojosa, Texas
Kenny Marchant, Texas                Carolyn McCarthy, New York
Duncan Hunter, California            John F. Tierney, Massachusetts
David P. Roe, Tennessee              Rush Holt, New Jersey
Glenn Thompson, Pennsylvania         Susan A. Davis, California
Tim Walberg, Michigan                Raul M. Grijalva, Arizona
Matt Salmon, Arizona                 Timothy H. Bishop, New York
Brett Guthrie, Kentucky              David Loebsack, Iowa
Scott DesJarlais, Tennessee          Joe Courtney, Connecticut
Todd Rokita, Indiana                 Marcia L. Fudge, Ohio
Larry Bucshon, Indiana               Jared Polis, Colorado
Trey Gowdy, South Carolina           Gregorio Kilili Camacho Sablan,
Lou Barletta, Pennsylvania             Northern Mariana Islands
Martha Roby, Alabama                 John A. Yarmuth, Kentucky
Joseph J. Heck, Nevada               Frederica S. Wilson, Florida
Susan W. Brooks, Indiana             Suzanne Bonamici, Oregon
Richard Hudson, North Carolina
Luke Messer, Indiana

                    Juliane Sullivan, Staff Director
                 Jody Calemine, Minority Staff Director

                            C O N T E N T S

                              ----------                              
                                                                   Page

Hearing held on September 10, 2013...............................     1

Statement of Members:
    Kline, Hon. John, Chairman, Committee on Education and the 
      Workforce..................................................     1
        Prepared statement of....................................     3
    Miller, Hon. George, senior Democratic member, Committee on 
      Education and the Workforce................................     3
        Prepared statement of....................................     5

Statement of Witnesses:
    Christie, Kathy, vice president, knowledge/information 
      management & dissemination, Education Commission of the 
      States.....................................................    24
        Prepared statement of....................................    26
    Kemple James J., executive director, Research Alliance for 
      New York City Schools, New York University.................    18
        Prepared statement of....................................    20
    Long, Dr. Bridget Terry, Ph.D., Xander professor of education 
      and economics, academic dean, Harvard Graduate School of 
      Education; chair, National Board for Education Sciences....     8
        Prepared statement of....................................    13
    Scott, George A., Director for Education, Workforce, and 
      Income Security Issues, U.S. Government Accountability 
      Office.....................................................     6
        Prepared statement of, Internet address to...............     8

Additional Submissions:
    Dr. Long:
        Document, ``Recommendations for the Reauthorization of 
          the Education Sciences Reform Act (ESRA)''.............    10
        Document, ``NBES Markup, June 2012, Education Sciences 
          Reform, Public Law 107-279,'' Internet address to......    13

 
                     EDUCATION RESEARCH: EXPLORING
                    OPPORTUNITIES TO STRENGTHEN THE
                    INSTITUTE OF EDUCATION SCIENCES

                              ----------                              

                      Tuesday, September 10, 2013

                     U.S. House of Representatives

                Committee on Education and the Workforce

                             Washington, DC

                              ----------                              

    The committee met, pursuant to call, at 10:01 a.m., in room 
2175, Rayburn House Office Building, Hon. John Kline [chairman 
of the committee] presiding.
    Present: Representatives Kline, Petri, Foxx, Walberg, 
Salmon, Rokita, Miller, Scott, Hinojosa, Tierney, Holt, 
Grijalva, Polis, and Bonamici.
    Staff present: Katherine Bathgate, Deputy Press Secretary; 
James Bergeron, Director of Education and Human Services 
Policy; Heather Couri, Deputy Director of Education and Human 
Services Policy; Lindsay Fryer, Professional Staff Member; 
Rosemary Lahasky, Professional Staff Member; Nancy Locke, Chief 
Clerk; Krisann Pearce, General Counsel; Dan Shorts, Legislative 
Assistant; Nicole Sizemore, Deputy Press Secretary; Alex 
Sollberger, Communications Director; Alissa Strawcutter, Deputy 
Clerk; Brad Thomas, Senior Education Policy Advisor; Tylease 
Alli, Minority Clerk/Intern and Fellow Coordinator; Jeremy 
Ayers, Minority Education Policy Advisor; Kelly Broughan, 
Minority Education Policy Associate; Jamie Fasteau, Minority 
Director of Education Policy; Scott Groginsky, Minority 
Education Policy Advisor; Eunice Ikene, Minority Staff 
Assistant; Brian Levin, Minority Deputy Press Secretary/New 
Media Coordinator; and Megan O'Reilly, Minority General 
Counsel.
    Chairman Kline. A quorum being present, the committee will 
come to order.
    Good morning. Welcome to our hearing to discuss 
opportunities to improve the Institute of Education Sciences.
    I would like to extend a special welcome to our witnesses, 
whose testimony will provide valuable insight into ways we can 
better ensure parents, teachers, school leaders, and 
policymakers have access to the most relevant education 
research.
    Established by the Education Sciences Reform Act in 2002, 
the Institute of Education Sciences is responsible for 
gathering information on education progress, conducting 
research on educational practices in the nation's schools, and 
examining the quality of federal education programs and 
initiatives. The information collected and disseminated by the 
institute helps schools identify and implement successful 
education initiatives.
    Additionally, the data allows taxpayers and congressional 
leaders to keep tabs on the federal investment in education, 
which is especially important in these times of fiscal 
restraint. The Education Sciences Reform Act has been due for 
reauthorization since 2008 and traditionally moves right after 
the Elementary and Secondary Education Act. In July the House 
approved the committee's legislation to rewrite ESEA, known as 
the Student Success Act.
    The Education Sciences Reform Act presents another 
opportunity to help provide teachers and parents the tools 
necessary to raise the bar in our schools and I look forward to 
working with my colleagues on both sides of the aisle to 
develop smart policies that will improve the law.
    To lay groundwork for the reauthorization, last year 
Ranking Member George Miller and I asked the Government 
Accountability Office to conduct a study on the effectiveness 
of the institute's research. Though the final report has yet to 
be released, we have received a few preliminary findings that 
highlight areas for improvement.
    For example, GAO confirms the institute has greatly 
improved the quality of education research over the last decade 
but notes there is often a significant delay in disseminating 
key data and findings to education officials. As a result, the 
research is not always immediately relayed to parents and 
school leaders, reducing its usefulness and relevancy.
    GAO also found the institute does not always properly 
evaluate the efficacy of its own programs and research arms, 
which could lead to unnecessary costs, confusion, and 
redundancies. Currently, the institute operates 10 regional 
labs and 12 research and development centers to conduct 
research, provide technical assistance, and distribute data. 
Meanwhile, the Department of Education operates five content 
centers and 16 comprehensive centers that serve some of the 
same purposes.
    As we develop policies to strengthen the institute, we 
should consider streamlining the federal research structure to 
reduce duplication, enhance accountability, and make it easier 
for states and school districts to access important 
information. We must also ensure the Institute of Education 
Sciences has the flexibility necessary to modernize its 
research methods and keep up with new developments in education 
delivery and practice. Finally, we must acknowledge that the 
value of the institute's research depends on its political 
autonomy and take the necessary steps to protect the 
organization's independence.
    We are fortunate to have with us several witnesses who can 
help us better understand what is and is not working with the 
Institute of Education Sciences, including a representative 
from GAO who can provide more information on the aforementioned 
study. Their testimony will inform our efforts to reauthorize 
the Education Sciences Reform Act and help us craft policies 
that will improve the quality and usefulness of education 
research.
    With that, I now yield to my distinguished colleague, 
George Miller, the senior Democratic member of the committee, 
for his opening remarks.
    [The statement of Chairman Kline follows:]

            Prepared Statement of Hon. John Kline, Chairman,
                Committee on Education and the Workforce

    Established by the Education Sciences Reform Act in 2002, the 
Institute of Education Sciences is responsible for gathering 
information on education progress, conducting research on educational 
practices in the nation's schools, and examining the quality of federal 
education programs and initiatives.
    The information collected and disseminated by the Institute helps 
schools identify and implement successful education initiatives. 
Additionally, the data allows taxpayers and congressional leaders to 
keep tabs on the federal investment in education, which is especially 
important in these times of fiscal restraint.
    The Education Sciences Reform Act has been due for reauthorization 
since 2008 and traditionally moves right after the Elementary and 
Secondary Education Act. In July the House approved the committee's 
legislation to rewrite ESEA, known as the Student Success Act. The 
Education Sciences Reform Act presents another opportunity to help 
provide teachers and parents the tools necessary to raise the bar in 
our schools, and I look forward to working with my colleagues on both 
sides of the aisle to develop smart policies that will improve the law.
    To lay groundwork for the reauthorization, last year Ranking Member 
George Miller and I asked the Government Accountability Office to 
conduct a study on the effectiveness of the Institute's research. 
Though the final report has yet to be released, we have received a few 
preliminary findings that highlight areas for improvement.
    For example, GAO confirms the Institute has greatly improved the 
quality of education research over the last decade, but notes there is 
often a significant delay in disseminating key data and findings to 
education officials. As a result, the research is not always 
immediately relayed to parents and school leaders, reducing its 
usefulness and relevancy.
    GAO also found the Institute does not always properly evaluate the 
efficacy of its own programs and research arms, which could lead to 
unnecessary costs, confusion, and redundancies. Currently the Institute 
operates 10 regional labs and 12 research and development centers to 
conduct research, provide technical assistance, and distribute data. 
Meanwhile, the Department of Education operates five content centers 
and 16 comprehensive centers that serve some of the same purposes.
    As we develop policies to strengthen the Institute, we should 
consider streamlining the federal research structure to reduce 
duplication, enhance accountability, and make it easier for states and 
school districts to access important information. We must also ensure 
the Institute of Education Sciences has the flexibility necessary to 
modernize its research methods and keep up with new developments in 
education delivery and practice. Finally, we must acknowledge that the 
value of the Institute's research depends on its political autonomy, 
and take the necessary steps to protect the organization's 
independence.
    We are fortunate to have with us several witnesses who can help us 
better understand what is and is not working within the Institute of 
Education Sciences, including a representative from GAO who can provide 
more information on the aforementioned study. Their testimony will 
inform our efforts to reauthorize the Education Sciences Reform Act and 
help us craft policies that will improve the quality and usefulness of 
education research.
                                 ______
                                 
    Mr. Miller. Thank you very much, Mr. Chairman, for holding 
this important hearing.
    As we begin the new school year our nation's schools stand 
on the threshold of major transitions. Schools are implementing 
new college and career ready standards, they are piloting new 
assessments aligned to those standards, and teachers and 
principals are being evaluated in new ways to measure and 
improve their effectiveness. Districts and states are 
implementing new accountability standards that focus intense 
efforts on turning around their most struggling schools.
    These are seismic changes. To ensure that children benefit, 
we need to support robust research to identify what is working 
and what can be improved.
    When Congress passed the Education Sciences Reform Act in 
2002 we did so to complement the bipartisan effort of No Child 
Left Behind. We wanted to strengthen the quality and the rigor 
of education research and we wanted to take advantage of the 
new, rich data that would emerge from the NCLB requirements.
    Eleven years later, thanks to NCLB and ESRA, we have a 
wealth of data that can be used to measure what is working for 
students, make corrections where things are not working, and 
create ways to ensure continuous improvement in the future. 
That is the task I hope this committee will take as we review 
ESRA.
    In 2002 the quality of education research was lacking. Much 
of it was driven by politics rather than science. As a 
response, we created the Institute of Education Sciences. Its 
mission was to conduct scientifically rigorous research outside 
the influence of politics or trend of the moment.
    Since then, IES has crafted high standards for the research 
it funds. It trains and supports researchers across the country 
and the studies are peer reviewed. A decade later, education 
research is more rigorous and sound.
    However, research is not effective if it only answers 
abstract questions or only published in professional journals. 
Research must be relevant as well as rigorous. It must be 
widely shared with those who work with students in order to 
make a difference.
    I am pleased that IES has taken steps in that direction. In 
2012 IES overhauled the nation's 10 regional research labs. It 
did so by connecting them with research alliances and 
policymakers and practitioners. The alliances work with the 
labs to identify pressing education problems in schools and 
then the labs develop and test strategies for solving them.
    Take New England, for example. The Northeastern regional 
lab created a research alliance on early childhood. That 
alliance gathered the region's early childhood stakeholders to 
create a research agenda that focuses on standards, 
assessments, and practices to improve early childhood 
education. The resulting research will now be more useful to 
practitioners that have a ready-made network for disseminating 
it.
    As we examine ESRA and the role of IES I hope that we will 
keep this need for both rigor and relevance in mind. I hope 
that we will keep in mind that we in Congress must be good 
federal partners in this effort. That means we must provide 
stable and sufficient resources to IES to do its job.
    The IES budget in 2012 fell just short of $600 million. 
That is less than 1 percent of our overall federal education 
budget. Other fields invest far more in research and 
development.
    We should take a serious look at this. More money is not 
always the answer, but sufficient money is.
    Also, we must not forget that sequestration cuts this year 
are limiting research right now, and another round of cuts 
beginning in January could have a crippling impact in 
destabilizing ongoing and vital research. I hope that my 
colleagues will join me in seeking ways to invest in education 
research in a smart way that avoids waste but also is sound and 
avoids harmful austerity.
    I thank all of the witnesses for appearing today and I am 
pleased that there is such an interest and leadership in 
addressing the quality of education research in America, and I 
look forward to all of your testimony.
    Thank you so much.
    [The statement of Mr. Miller follows:]

  Prepared Statement of Hon. George Miller, Senior Democratic Member, 
                Committee on Education and the Workforce

    Chairman Kline, thank you for holding this important hearing. As we 
begin a new school year, our nation's schools stand on the threshold of 
major transitions.
    Schools are implementing new college and career ready standards. 
And they are piloting new assessments aligned to those standards. 
Teachers and principals are being evaluated in new ways to measure and 
improve their effectiveness. Districts and states are implementing new 
accountability systems that focus intense efforts on turning around 
their most struggling schools.
    These are seismic changes. To ensure that children benefit, we need 
to support robust research to identify what is working and what can be 
improved.
    When Congress passed the Education Sciences Reform Act in 2002, we 
did so to complement the bipartisan effort of No Child Left Behind. We 
wanted to strengthen the quality and rigor of education research. And 
we wanted to take advantage of the new, rich data that would emerge 
from NCLB requirements.
    Eleven years later, thanks to NCLB and ESRA, we have a wealth of 
data that can be used to measure what is working for students, make 
corrections where things are not working, and create ways to ensure 
continuous improvement in the future. That is the task I hope this 
Committee will take up as we review ESRA.
    In 2002, the quality of education research was lacking. Much of it 
was driven by politics rather than science. As a response, we created 
the Institute of Education Sciences. Its mission was to conduct 
scientifically rigorous research, outside the influence of politics or 
the trend of the moment.
    Since then, IES has crafted high standards for the research it 
funds. It trains and supports researchers across the country. Its 
studies are peer reviewed. A decade later education research is more 
rigorous and sound.
    However, research is not effective if it only answers abstract 
questions or is published in a professional journal. Research must be 
relevant as well as rigorous. And it must be widely shared with those 
who work with students, in order to make a difference.
    I am pleased that IES has taken some steps in this direction. In 
2012 IES overhauled the nation's 10 regional research labs. It did so 
by connecting them with research alliances of policymakers and 
practitioners. The alliances work with the labs to identify pressing 
education problems in schools and then the labs develop and test 
strategies for solving them.
    Take New England, for example. The Northeastern regional lab 
created a research alliance on early childhood. That alliance gathered 
the region's early childhood stakeholders to create a research agenda 
that focuses on standards, assessments, and practices to improve early 
education. The resulting research will now be more useful to 
practitioners and have a ready-made network for disseminating it.
    As we examine ESRA and the role of IES, I hope we will keep this 
need for both rigor and relevance in mind. I hope we also keep in mind 
that we in Congress must be good federal partners in this effort. That 
means we must provide stable and sufficient resources to IES to do its 
job.
    The IES budget for FY12 fell just short of $600 million. That is 
less than 1 percent of our overall federal education budget. Other 
fields invest far more in research and development. We should take a 
serious look at this. More money is not always the answer, but 
sufficient money is.
    Also, we must not forget that sequestration cuts this year are 
limiting research right now. And another round of cuts beginning in 
January could have a crippling impact and destabilize ongoing and vital 
research. I hope my colleagues will join me in seeking ways to invest 
in education research in a smart way that avoids waste; but also in a 
sound way that avoids harmful austerity.
    I thank all the witnesses for appearing today. I am pleased that 
there is such interest and leadership in addressing the quality of 
education research in America.
    I look forward to your testimony. I yield back.
                                 ______
                                 
    Chairman Kline. Thank the gentleman.
    Pursuant to committee rule 7(c), all committee members will 
be permitted to submit written statements to be included in the 
permanent hearing record, and without objection the hearing 
record will remain open for 14 days to allow statements, 
questions for the record, and other extraneous material 
referenced during the hearing to be submitted in the official 
hearing record.
    It is now my pleasure to introduce our very distinguished 
panel of witnesses.
    First, Mr. George A. Scott is the director for education, 
workforce, and income security with GAO. He has previously 
testified before both the House and Senate on the agency's work 
surrounding K-12 education and student financial aid programs.
    Dr. Bridget Terry Long is the Xander professor of education 
and economics and academic dean at the Harvard Graduate School 
of Education. She is also chair of the National Board for 
Education Sciences.
    Dr. James Kemple is executive director of the Research 
Alliance for New York City Schools at New York University.
    And Ms. Kathy Christie is the vice president for knowledge 
and information management and dissemination at the Education 
Commission of the States.
    Welcome, all of you. Before I recognize each of you to 
provide your testimony let me remind you of our very nifty 
lighting system here.
    You will each have 5 minutes to present your testimony. 
When you begin the light will turn green--no surprises here. 
When you have a minute left the light will turn yellow, and 
when your time is expired the light will turn red.
    And I am reluctant to gavel a witness down but I would ask 
you to try to wrap up at that point so that each of you has a 
chance to provide your testimony and we have a chance to engage 
in a discussion.
    I will now like to recognize Mr. Scott for 5 minutes.
    Sir, you are recognized.

     STATEMENT OF GEORGE A. SCOTT, DIRECTOR FOR EDUCATION, 
    WORKFORCE, AND INCOME SECURITY ISSUES, U.S. GOVERNMENT 
                     ACCOUNTABILITY OFFICE

    Mr. Scott. Thank you.
    Chairman Kline, Ranking Member Miller, and members of the 
committee, thank you for inviting me here today to discuss the 
preliminary results of our review of the Department of 
Education's Institute of Education Sciences, IES.
    The Education Sciences Reform Act outlines a broad mission 
for IES, including to expand the knowledge and understanding of 
education and to provide this information to a wide range of 
stakeholders. My testimony will focus on the extent to which 
IES supports high quality research, disseminates relevant 
products to the education field, and coordinates within 
education and with other federal agencies.
    In summary, we found that since its creation IES has 
substantially improved education research. In 2007 the Office 
of Management and Budget concluded that IES had transformed the 
quality and rigor of research within the Department of 
Education. It had also increased the demand for scientifically 
based evidence of effectiveness in the education field as a 
whole.
    Many stakeholders said that IES's research standards 
improved the quality of and had a positive influence on 
education research.
    While IES has improved the quality of education research, 
its research is sometimes of limited usefulness to policymakers 
and practitioners. Some stakeholders told us that the 
evaluations supported by IES may not be completed soon enough 
to inform important policy decisions. For example, officials 
from one organization said that IES's evaluation of the Race to 
the Top and school improvement grant programs will not be 
released in time to give the states opportunity to implement 
lessons learned from these studies before funding for these 
programs end.
    To address concerns about the relevance of its research IES 
is soliciting feedback from stakeholders. For example, IES 
recently convened a group of state and local education 
officials to discuss the strengths and weaknesses of the 
regional education labs and the What Works Clearinghouse.
    Despite these efforts, IES does not have a systematic 
process for incorporating feedback from stakeholders into its 
research agenda. We consider having such a process to be a key 
element of promoting a sound federal research program.
    Further, IES cannot demonstrate the impact of its efforts 
to improve the quality and relevance of its research in some 
areas. IES's performance measures no longer capture the full 
range of IES research and priorities. In some cases these 
measures are no longer relevant to managing the agency's 
operations.
    Additionally, IES does not publicly report on the 
performance of the regional education labs, which constitute 
one of the agency's largest investments. As we have previously 
reported, without appropriate performance measures agencies may 
be at risk for failing to achieve their stated goals.
    IES's research and technical assistance group have taken 
various steps to provide relevant research. They have engaged 
policymakers and practitioners in planning research and 
technical assistance activities. All three groups also use a 
range of methods to disseminate their products.
    Despite these efforts, stakeholders have raised concerns. 
For example, some stakeholders said that they do not find 
research by the regional education labs to be as relevant or as 
timely as other sources of information. Stakeholders also noted 
that further efforts are needed to better market the research 
from these groups so that they reach intended audiences.
    Finally, IES takes a number of steps to coordinate with 
other federal agencies to increase the use of research evidence 
in guiding funding decisions. IES also coordinates within 
Education to facilitate collaboration among various research 
programs.
    However, despite these efforts, the Department of Education 
faces challenges in funding and prioritizing evaluations. 
According to Education officials, efforts to prioritize 
evaluation projects are hindered in part because of statutory 
requirements.
    According to these officials, Education lacks the authority 
to combine evaluation funds from programs across the department 
and then use them to evaluate any other program. As a result, 
some evaluations may not occur and high priority evaluations 
may be delayed.
    In conclusion, IES has made significant contributions to 
strengthening the rigor of education research. However, it 
could build on these efforts by continuing to improve its 
ability to provide timely and relevant information.
    In addition, with a systematic process for incorporating 
stakeholder needs and more comprehensive and up-to-date 
performance measures IES would position itself to more fully 
achieve its mission. Also, the ability to prioritize 
evaluations is critical to helping the Congress make informed 
decisions about programs. As we complete our work we will 
consider any recommendations needed to address these issues.
    Mr. Chairman, this concludes my remarks. Thank you.
    [The statement of Mr. Scott may be accessed at the 
following Internet address:]

                http://www.gao.gov/assets/660/657669.pdf

                                ------                                

    Chairman Kline. Thank you very much.
    Dr. Long, you are recognized.

   STATEMENT OF DR. BRIDGET TERRY LONG, XANDER PROFESSOR OF 
EDUCATION AND ECONOMICS, ACADEMIC DEAN, HARVARD GRADUATE SCHOOL 
  OF EDUCATION, CHAIR, NATIONAL BOARD FOR EDUCATION SCIENCES, 
                INSTITUTE OF EDUCATION SCIENCES

    Ms. Long. Thank you, and good morning.
    Chairman Kline and members of the committee, thank you for 
the opportunity to appear before you today. As noted, I am also 
chair of the National Board for Education Sciences.
    The board is independent of IES and we are tasked with 
advising the IES director and reviewing and evaluating its 
work. In this way, we provide a critical but also constructive 
perspective on the activities of IES.
    My testimony reflects the discussions and recommendations 
by the board as well as my observations as an experienced 
researcher.
    As we work to raise student achievement, foster productive 
learning environments, and bolster the social contributions of 
our schools and universities, the knowledge, inventions, and 
partnerships created through education research are essential. 
It is through research that we determine the best ways to 
produce the needed gains and help to make tough decisions about 
how to use our limited funds.
    During the short history of IES it has filled an essential 
role in providing and encouraging the necessary conditions for 
high quality research in education. I focused my comments on 
three main contributions.
    First, IES has taken the role of creating a series of 
important public goods. By ``public goods'' I mean things that 
benefit us all, but many of these goods would not have been 
produced without government intervention.
    As noted by Chairman Kline, IES provides the foundations of 
factual information and research with the collection of clear, 
consistent, high quality data. Additionally, IES serves as a 
repository and distribution center of objective research.
    This is vitally important because the education space is 
filled with many organizations, companies, and individuals who 
have varying objectives, agendas, and degrees of expertise. 
Therefore, it can be difficult to discern between the many 
studies, reports, and assertions of what is fact versus what is 
fiction, and IES stands as the best authority of rigorous 
research free from influence.
    Second, IES has led the way in efforts to reevaluate and 
redefine the standard of what is considered the best evidence. 
Before the creation of IES many lamented that education 
research was failing to answer important educational questions 
in convincing ways. One example of what IES has done is push 
for randomized control trials, which is a gold standard in 
research and often used in the field of medicine.
    By providing support and encouraging researchers to develop 
ways of conducting RCTs while still being sensitive to the 
needs of students, education research has progressed in 
fundamental ways with new, important evidence on the effects of 
key programs and interventions. Moreover, IES continues to 
engage the field in conversations about rigor, as demonstrated 
by technical work groups tasked with ensuring that evaluations 
provide unbiased assessments.
    Third, IES has influenced the kind of research that is 
done. While there are many organizations that conduct education 
research, most focus on only a handful of topics and are only 
able to do projects of limited size. But education is all 
encompassing, from the wide array of the types of students, 
environments, needs, and goals, and there is much work to be 
done.
    With a national platform, IES has the unique ability to 
leverage researcher and practitioner expertise by signaling and 
providing incentives to conduct studies on issues of importance 
for the country. This includes large-scale projects that would 
not otherwise be conducted but shed an considerable light on 
important issues.
    Another contribution has been to emphasize the importance 
of partnerships with researchers and schools, districts, and 
educational agencies. By working closely with the field, 
researchers are much more likely to produce research that is 
relevant and useful in practice.
    Finally, it is important to note that IES has been 
instrumental in attracting talent to the study of education 
with training, tools, and resources to support high quality 
research.
    While the accomplishments of IES are numerous, the board 
and IES are committed to continuous improvement. As noted 
earlier, IES has filled a gap for the nation by providing 
clear, objective information. However, more could be done to 
communicate and disseminate this information.
    This is a challenging feat. Education has an incredibly 
large range of stakeholders and multiple audiences to address, 
including policymakers, practitioners--from teachers, to 
superintendents, to state agencies--researchers, and students 
and their families. Each group needs different kinds of 
information in different forms.
    There are many examples of success and promise. For 
example, the practice guides distill a wealth of research into 
clear steps for teachers to take to improve the learning of 
students.
    Additionally, there have been many efforts to improve the 
work of IES. They include revisions to the Web site, 
establishing a grant competition to create a research and 
development center on knowledge utilization. To ensure reliance 
and usability IES has also revised and renegotiated the 
contracts for the RELs and just last month convened a product 
feedback and development meeting with stakeholders from across 
the country.
    I have also entered into the record a full list of the 
board's recommendations regarding ESRA.
    [The information follows:]

            THE NATIONAL BOARD OF EDUCATION SCIENCES (NBES)

 Advisory Board to the Director of the Institute of Education Sciences 
                                 (IES),
       U.S. Department of Education Dr. Bridget Terry Long, Chair

             Recommendations for the Reauthorization of the
                  Education Sciences Reform Act (ESRA)

    At the June 20, 2012 NBES meeting, Board members discussed 
recommendations to revise ESRA. These recommendations build from 
previous suggestions made by the NBES in May 2008 with several 
additional changes and revisions. The recommendations fall into three 
categories:
     Definitional changes
     Substantive changes in the Institute or Board's 
functioning or powers
     Administrative or ``housekeeping'' changes to the bill
                          definitional changes
1. Definitions related to ``Scientific Research,'' etc.
    On pages 4-5, we recommend changes related to definitions of 
``scientific research.'' Congress has moved towards defining principles 
of scientific research rather than defining scientifically-based 
research. As noted on page 5, the NBES agrees with the Department of 
Education's position that any definition of ``scientific research,'' 
etc. in ESRA should be consistent with the definitions used in other 
bills such as the Elementary and Secondary Education Act.
2. Changes in IES's mission
    On page 7, the NBES recommends modifying the initial lines of IES's 
mission to read:
    The mission of the Institute is to provide national leadership in 
expanding reliable evidence on which to ground education practice and 
policy and to encourage its use by parents, educators, students, 
researchers, policymakers, and the general public * * *
    The major differences between our recommendation and the original 
are to replace ``expanding fundamental knowledge'' with ``expanding 
reliable evidence'' and to add the words ``encourage its use''. The new 
definition, which focuses on providing evidence and encouraging its 
use, is much closer to what IES does and is more objective than the 
existing definition's focus on fundamental knowledge and understanding.

    Note: This is not marked on the attached draft.
 substantive changes related to powers, functions, and terms of office
1. Delegation of ``Other Activities''
    Changes on page 9 would leave it at the discretion of the IES 
Director to accept additional assignments from the Secretary of 
Education if they were consistent with IES's mission and priorities. 
The existing language gives the Secretary the power to simply assign 
such activities to IES.
2. Provisions related to the IES Director
    Page 9 removes language which is no longer relevant related to the 
appointment of the first IES Director. In addition, language is added 
making it possible for a Director to be nominated for a second term, 
and for a sitting Director to serve up to an additional year if his/her 
successor has not been appointed.
    Additional new language on page 9-10 makes the IES Director 
eligible for ``critical pay'' under the Federal Workforce Flexibility 
Act of 2004. The explanatory note in the mark-up states, ``Many people 
who might be qualified to be Director are unwilling to do so over a 6 
year term at the rate of pay [specified in existing law]. This addition 
provides pay flexibility in recruiting a director, and would be subject 
to the recommendation of the Board.'' Page 13 adds the language related 
to the Board's ability to make recommendations in this area.
    New language on page 10 specifies that the IES Director reports 
directly to the Secretary of Education. The explanatory note in the 
mark-up states, ``A direct reporting line to the Secretary is important 
to maintaining the status and independent functioning of IES within the 
Department.''
3. Requirement that IES Director submit a biennial plan of activities 
        to the Board for advice
    Page 10 adds new language to the Director's duties requiring him/
her to submit a biennial plan of activities to the Board every two 
years. New language on page 12 adds reviewing and advising the Director 
on the plan of activities to the Board's duties. (Note that the Board's 
approval of the plan is not required.)
4. NBES: Organizations that advise the President on Board members
    Existing language in ESRA requires the President to solicit advice 
regarding individuals to serve on the Board from the National Academy 
of Sciences, the National Science Board, and the National Science 
Advisor. New language on page 13 would add the Board itself, the 
American Educational Research Association, the Society for Research on 
Educational Effectiveness, and the National Academy of Education to the 
list of groups that the President must solicit advice from.
5. NBES: Board terms
    Page 14 includes a number of changes aimed at fixing some of the 
difficulties that the Board has consistently encountered since its 
founding, including numerous vacancies and attenuated terms.
    Page 14 adds language specifying that a member's 4-year term 
commences from the date of their appointment. Current practice has been 
to appoint members to 4-year slots, whose beginning and end dates are 
calculated based on three cohorts tied to the original legislation. 
That is: there are five Board slots tied to an initial term expiration 
date of 11/28/08; five Board slots tied to an initial term expiration 
date of 11/28/06; and five Board slots tied to an initial term 
expiration date of 11/28/07. If a Board member was nominated or 
confirmed to a slot that was close to its expiration date, their 
effective term dating from their actual appointment might be as short 
as a year. Page 14 also strikes language which is no longer relevant 
related to the initial appointment of Board members to staggered terms.
    Page 14 also adds a new provision (similar to the provision for the 
IES Director) allowing Board members to serve up to an additional year 
after their term has expired if their successor has not been appointed.
    The net effect of the changes listed on page 14 will be to greatly 
reduce the number of unfilled vacancies on the Board and to make it 
much more likely that at all times the Board will have close to its 
full complement of 15 members that ESRA stipulates.
6. NBES: Executive Director
    New language on page 15 of the mark-up provides greater detail 
regarding the Executive Director position. Board members favored 
revising ESRA to give the Board hiring and evaluation authority over 
the NBES Executive Director.
7. NBES: Charitable contributions
    New language on page 16 would allow the Board to accept charitable 
donations to further the mission of the Board. This would allow the 
Board to provide coffee during advisory board meetings.
8. NBES: Standing committee structure
    Pages 16 strikes language related to NBES's standing committee 
structure. The existing language specifies a standing Board committee 
corresponding to each national education center (e.g., NCES, NCER). The 
Board has never functioned this way in practice. This is due, in part, 
to the fact that at times Board membership has dwindled to as few as 6 
members.
    In place of the struck language, the mark-up adds new permissive 
language on page 16 that allows the Board to establish standing 
committees related to the Board's responsibilities.
9. Commissioner's pay
    Similar to the new language related to the IES Director's pay, page 
18 adds language allowing Commissioners to be eligible for critical pay 
under the provisions of Federal Workforce Flexibility Act of 2004. The 
rationale is similar to that for the IES Director: to enhance 
recruitment flexibility for Commissioners.
10. The appointment process for the NCES Commissioner
    Page 18 has existing language that has the NCES Commissioner be 
appointed by the President and confirmed by the Senate. The Board made 
no recommendation regarding changing to the appointment process for the 
NCES Commissioner, because opinion was evenly divided among the 
members. Some felt strongly that the current requirement that the NCES 
Commissioner be appointed by the President and confirmed by the Senate 
reflects a hard-won acknowledgment that education statistics deserve 
national-level prestige. Others instead felt that the procedure for 
appointing the NCES Commissioner should be the same as that for the 
other IES Center commissioners to support smooth and efficient 
functioning of IES.
11. National Research and Development Centers
    Pages 21-22 strike existing language related to National Research 
and Development Centers and replaces it with a section titled, 
``Priorities for Long-Term Research Activities.'' The changes remove 
language requiring the funding of at least eight National R&D Centers 
as well as requirements related to the topics assigned to the centers. 
The replacement language does not address the number of centers funded 
and allows the NCER Commissioner to choose topics consistent with IES's 
priorities. The feeling is that the NCER commissioner and the director, 
with the counsel of the Board, should be able to determine the best 
funding mechanisms and funding levels for advancing IES's long-term 
research priorities rather than having Congress earmark particular 
centers and levels of funding.
12. Removal of privacy protection for individual school information
    Page 41 strikes language giving privacy protection to individually 
identifiable information with respect to individual schools. The 
explanatory note in the mark-up states that, ``Schools do not receive 
privacy protection elsewhere in federal statute or regulations. Many 
IES reports from NCES require that schools be identified, e.g., the 
Common Core of Data. The prohibition on revealing school identity means 
that useful information must be omitted from evaluation reports. There 
is no compelling reason to maintain this protection for schools.''
13. Adjustment to the circumstances under which the Director or Board 
        members may be removed
    Pages 45-46 add language that allows the President to remove the 
Director and any Board member for cause, although the President must 
inform the Board of the cause for which the appointee is being removed. 
The original language did not include the words ``for cause,'' nor was 
Board notification required. The original language also included the 
Commissioner for Education Statistics in these provisions. However, the 
mark-up's proposed change to make the Commissioner of Education 
Statistics a Director-appointed position, like the other Commissioners, 
means that the Commissioner should be struck from the provisions of 
this section.
14. Expansion of authorization related to data bases to be included in 
        the statewide longitudinal data systems
    Page 52 adds language specifying the Higher Education Act and IDEA 
with regard to the development of statewide longitudinal data systems. 
The existing language only specifies the Elementary and Secondary 
Education Act. The change expands the authority so that a broader range 
of educational records can be incorporated into the supported data 
systems.
15. NAEP reports
    Pages 55 and 57 add language related to authority over the content 
and release of National Assessment of Educational Progress (NAEP) 
reports. The mark-up's explanatory note states, ``There are ongoing 
disputes between NCES and the National Assessment Governing Board 
(NAGB) on the content and formatting of NAEP reports * * * These are 
IES/NCES reports published under the authority of the Director and 
Commissioner. The NCES commissioner needs to retain responsibility for 
the content of the reports.'' And, ``NAGB has taken the position that 
the NCES commissioner's role at the release event is entirely at the 
discretion of NAGB. Because the findings being released are from an 
NCES statistical report, the commissioner or his delegate should be 
responsible for presenting the findings.''
                  administrative/housekeeping changes
1. Delegation of authority
    Page 8 contain new language aimed at delineating the delegation of 
authority between the IES Director and the Secretary of Education.
2. Removal of language related to the role of the National Assessment 
        Governing Board (NAGB)
    Four provisions related to NAGB are struck on pages 8-9. The 
rationale given was that a preceding paragraph clearly articulated 
NAGB's role and so the subsequent four provisions were redundant.
3. Other changes in the Director's duties
    Page 10 changes language related to peer review from ``establish'' 
to ``maintain,'' reflecting the fact that peer review procedures have 
already been established.
    Another change on page 11 specifies that the Director will 
coordinate with the Secretary of Education to insure that IES's 
findings are used by all of ED's technical assistance providers, and 
not just the 15 comprehensive assistance centers.
4. Review of publications, not ``products''
    Page 11 strikes the term ``products'' from the section pertaining 
to the Director's review of evidence-based claims in ED publications. 
The rationale is that IES cannot review products, only publications 
that make scientific claims.
5. Requirement that IES priorities be proposed every 6 years
    New language on page 11 requires the IES Director to submit 
priorities for the Institute to the Board for approval at least every 6 
years. This would put into statute what has occurred in practice.
6. Peer review standards and NCER
    Page 19 strikes language requiring the National Center for 
Education Research (NCER) to maintain peer review standards. This would 
conform to IES's actual practice, which is to have the Scientific 
Review Office maintain IES's standards related to peer review.
7. Replacing `Commissioner' with `Center'
    Pages 26 and 29 replace `Commissioner' with `Center' for the sake 
of consistency.
8. Removal of outdated language related to NCEE
    Page 32 strikes language regarding the award of specific contracts 
which is outdated and not relevant for reauthorization. Similarly, 
pages 34-35 strikes additional outdated language that is not relevant 
to reauthorization.
                                 ______
                                 
    [An additional submission, the document, ``NBES Markup, 
June 2012, Education Sciences Reform, Public Law 107-279,'' may 
be accessed at the following Internet address:]

        http://ies.ed.gov/director/board/pdf/NBESmarkupESRA.pdf

                                ------                                

    Ms. Long. So in summary, IES has made many contributions, 
though there is still work to be done, and we look forward to 
this discussion.
    Thank you.
    [The statement of Dr. Long follows:]

 Prepared Statement of Dr. Bridget Terry Long, Ph.D., Xander Professor 
 of Education and Economics Academic Dean, Harvard Graduate School of 
        Education; Chair, National Board for Education Sciences

    Mr. Chairman and members of the committee, thank you for the 
opportunity to appear before you today.
    My name is Dr. Bridget Terry Long, and I am the Academic Dean and 
Xander Professor of Education and Economics at the Harvard Graduate 
School of Education. Beyond my expertise as a researcher and faculty 
member, I am also the Chair of the National Board for Education 
Sciences, the advisory board of the Institute of Education Sciences 
(IES). The National Board for Education Sciences is independent of IES, 
and we are tasked with advising the Director and reviewing and 
evaluating the work of IES. In this way, we provide a critical but also 
constructive perspective on the activities of IES.
    My testimony reflects discussions and recommendations made by the 
Board as well as my observations as an experienced educational 
researcher who has interacted with IES on many levels. My comments 
today aim to provide an objective assessment of the role of IES, its 
contributions, and areas for improvement.
The Role of IES
    In our current environment, educational research has become even 
more important as the penalties of poor achievement and lack of 
opportunity have never been greater. As we work to raise student 
achievement, foster productive learning environments, and bolster the 
social contributions of our schools and universities, the knowledge, 
inventions, and partnerships created through educational research are 
essential--it is through research that we determine the best ways to 
produce the needed gains and help to make tough decisions about how to 
use our limited funds. Before we can debate what policies we should 
implement, we first need a clear understanding of the facts and to have 
an accurate sense of the real costs and benefits of any policy or 
program. In essence, research is the foundation for improving 
education.
    During the short history of IES, it has filled an essential role in 
providing and encouraging the necessary conditions for high-quality 
education research. While its impact is evident in many ways, I focus 
my comments on three main contributions. First, IES has taken the role 
of creating a series of public goods that no one else would or could do 
without concerns about possible bias. Second, it has led the way in 
efforts to reevaluate and redefine the standard of what is considered 
good evidence. Third, IES has influenced the kind of educational 
research that is done by making possible large-scale studies, pushing 
researchers to work closely with practitioners to ensure relevance and 
usability, and holding an unwavering focus on serving the national 
good.
            (1) Creating Necessary Public Goods
    As a federal entity, IES has taken leadership to provide several 
key public goods needed to support a strong educational system and 
research. By public goods, I mean things that benefit us all, but many 
of these goods would not otherwise be produced without government 
intervention. For example, IES provides the foundations of factual 
information and research with the collection of clear, consistent, 
high-quality data through the National Center for
    Education Statistics (NCES).\1\ It is through the efforts of IES, 
which conducts its work free from political influence, that we are able 
to understand trends in our student populations, schools and 
universities, and an array of inputs and outcomes that span early 
childhood to adult education. These data also make possible a wealth of 
research conducted on every aspect of education.
    Additionally, IES serves as a repository and distribution center of 
research, both studies funded by IES and those that are not. The 
dissemination and communication of objective information is a critical 
one for the nation. The education space is filled with many 
organizations, companies, and individuals who have varying objectives, 
agendas, and degrees of expertise. Therefore, it can be difficult to 
sort between the many studies, reports, and assertions to determine 
what is fact versus what is fiction. Moreover, the research community 
often lacks the training and incentives to translate complex research 
for a lay audience. In such a crowded space, IES stands as the best 
authority of rigorous research free from influence. It has helped to 
clarify what is known about issues related to large educational 
debates. Moreover, it has been helpful in discerning between 
conflicting and confusing reports on important issues. It has used its 
convening power to bring together researchers from various backgrounds 
to discuss the issues and coordinate research.\2\ It has also conducted 
evaluations of federal initiatives.\3\
            (2) Setting the Standards of ``Good Evidence''
    Before the creation of IES, many lamented that educational research 
was failing to answer important questions in convincing ways. The 
varying quality of research and lack of attention to certain issues led 
some to dismiss the educational research base as inadequate. IES has 
changed this dramatically by leading a critical assessment of past 
research and initiating a number of debates about what are appropriate 
methods and standards of rigor for the different approaches to 
educational research.
    One concrete example of this has been the push for randomized 
controlled trials (RCTs), which are considered the gold-standard of 
research and often used in the field of medicine. Prior to IES's 
leadership, RCTs were rarely conducted in education and not valued 
among many researchers. However, by pushing the field, providing 
support, and engaging researchers to develop ways of conducting such 
analyses while still being sensitive to needs of students and 
practitioners, educational research has progressed in fundamental ways 
with new important evidence on the effects of key programs and 
interventions. For example, in my own work with several colleagues, 
which was partially funded by IES, we demonstrated that providing low- 
and moderate-income families with streamlined personal assistance to 
complete the federal college financial aid application had large 
effects on college attendance and persistence. Because we used a 
randomized controlled trial design, we were able to establish 
convincingly that our intervention was not only the cause of the 
educational gains; importantly, the program was also inexpensive.\4\
    IES continues to engage the field in conversations about rigor in 
educational research. This is demonstrated by technical working groups 
that are establishing standards for specific research methodologies and 
helping to ensure that evaluations provide unbiased and causally-valid 
assessments.\5\ It is also worth noting that IES has developed a 
rigorous peer review process for evaluating grant proposals.
            (3) Encouraging Relevant, Rigorous Research for the 
                    National Good
    IES has used its resources and convening power to focus the field 
on research that is both rigorous and focused on shedding light on the 
major problems facing the country. By setting priorities and crafting 
calls for research proposals (i.e., Requests for Proposals or RFPs), 
IES has sent signals to the field about important topics that need 
answers, rigorous standards that must be upheld, and the importance of 
conducting research in partnership with practitioners.
    Additionally, it has made possible research studies that would not 
have otherwise been conducted.
    While there are private foundations and other organizations that 
support educational research, most focus on only a handful of topics 
and fund projects of limited size. But education is all encompassing, 
from the wide array of types of students, environments, needs, and 
goals, and there is much work to be done. With a national platform, IES 
has the unique ability to leverage researcher and practitioner 
expertise by signaling and providing incentives to conduct studies on 
issues of importance for the country. One way it has done this is by 
designing research competitions that focus on the major issues and 
areas of education. Along with this has come IES's emphasis on the 
importance of external validity in research, meaning that it has called 
for researchers to be accountable to external audiences on how the 
findings for one set of schools might be applicable to another set of 
schools.
    IES has also been able to support large-scale projects that could 
not be easily funded by others. To learn more certain issues, studies 
must be large in scale and compare the experiences of districts across 
states or large populations of students. Without support from IES, this 
type of work would often not be possible, and the knowledge base that 
is being built as a result of this work has been valuable in improving 
student outcomes. Taken together, IES has both insured research on a 
breadth of topics while also making possible large-scale studies that 
have been incredibly beneficial to our understanding of how to help 
students.
    Another way IES has influenced the research community is by 
highlighting the importance of partnerships between researchers and 
schools, districts, or state educational agencies. Because the delivery 
of education is the result of many actors, research can often be 
improved by being designed and conducted while working with 
practitioners. Additionally, by working closely with the field, 
researchers are much more likely to produce research that is relevant 
and useful in practice. However, such work can be difficult to manage 
and implement.
    IES has pushed and supported such connections to the benefit of the 
research being conducted.\6\
    Finally, it is important to note that IES has been instrumental in 
attracting talent to the study of education. With the signals it sends 
about important issues in education and the support it gives for 
research, IES has helped to attract a growing number of researchers 
with the tools and resources to support high-quality research and 
partner with the field. IES is helping to produce the next generation 
of scholars and innovators who will help to solve important problems in 
education.
The Strengths, Challenges, and Continuous Improvement of IES
    The accomplishments of IES are numerous, and the researchers and 
innovators supported by IES funding will continue to have positive 
impacts on the lives of students as well as many other parts of our 
society. Nevertheless, in light of the Board and IES's commitment to 
continuous improvement, it is clear more can and needs to be done. In 
this spirit, the Board has worked to advise, review, and advance the 
activities of IES. The Board has matured to be an important place of 
feedback and expertise, and my comments here reflect continuing 
discussions between the Board and IES staff about how to address 
challenges facing the organization.
    As I noted earlier, the dissemination and communication role of IES 
is an important one.
    IES has filled a gap for the nation by providing clear, objective 
information and making it available to the public. While IES is a 
strong producer and supporter of information of value, it is still 
building capacity and expertise on how to disseminate that information, 
including methods that use the latest technology and outreach methods. 
This is a challenging feat. Unlike many other fields, education has 
large range of stakeholders and multiple audiences to address, 
including policymakers; practitioners from teachers to superintendents 
to state agencies; researchers; and students and their families. Each 
group needs different kinds of information in different forms.
    The Board and IES staff believe strongly in the dissemination role 
of IES, and we have held a number of discussions on how to improve 
efforts. There are many examples of success and promise. For example, 
the Practice Guides distill a wealth of research into clear steps 
teachers can take to improve the learning of their students.\7\ The 
What Works Clearinghouse was created with the idea of helping the 
public understand research results and whether they were completed 
using rigorous methods. The dissemination of recent data reports and 
grant competitions include webinars and video media.\8\
    However, more could be done in terms of reaching out to the many 
audiences of educational data and research, and there are many efforts 
underway at IES to address this challenge. They include:
     Revisions to the website to make it easier to find 
important research and facts. For instance, a new contract was awarded 
this year to manage and enhance the What Works Clearinghouse.\9\ 
Additionally, as part of the RFP for the Education Resources 
Information Center (ERIC), the contractor is expected to redesign the 
IES website to improve search capabilities and provide basic 
orientations to key topics and references for relatively inexperienced 
users.\10\
     IES added new requirements to research grant competitions 
for researchers to develop dissemination plans for their studies. 
Moreover, NCSER released a report on how to make research more 
understandable, and it was presented to its grant recipients.\11\
     Establishing a grant competition to create a Research and 
Development Center on Knowledge Utilization. This Center will explore 
questions of how education researchers can make their work more 
relevant and useful to practitioners located in state and local 
education agencies and in individual schools. This work is meant to 
address concerns that often there is only limited adoption of evidence-
based practices.\12\
    Related to the issue of dissemination is the relevance and 
usability of the research produced and funded by IES. This has been a 
major focus of IES, and there are many instances of the Institute 
meeting this goal. As noted above, the growing attention to the 
importance of partnerships has broadened the number of studies done in 
concert with schools and districts, and this approach increases the 
likelihood that the results will be relevant and useful for 
practitioners. Still, this has been an area of constant reevaluation, 
and there have been many activities recently to improve this function 
of IES. For example:
     Revising and renegotiating the contracts for the Regional 
Education Labs (RELs). For example, earlier this year, IES released 
revised criteria for REL proposals and products. The criteria focus on 
issues related to the technical rigor of products (e.g., data quality, 
analysis methods), the relevance of the work (i.e., whether it provides 
evidence that can inform a practitioner's action or decision), and the 
readability of the products (i.e., whether the information is clear for 
its intended audiences). NCEE has also been working to build the 
capacity of the REL program by conducting webinars to help the RELs 
meet increasing standards in writing, collaboration, and 
measurement.\13\
     Just recently, on August 12, 2013, IES convened a Product 
Feedback and Development Meeting with stakeholders from across the 
country to get suggestions about how to improve the usability and 
relevance of the products and services of the WWC and RELs.\14\
    As an independent body tasked with providing constructive feedback 
to IES, the Board has been pleased with the fact that our feedback and 
that of others on these issues has been incorporated into the work of 
the Institute, and we believe these activities will help to strengthen 
IES's impact.
    Another challenge facing IES is balancing the need to work in many 
areas with the reality of having limited resources. Because it is 
important to understand so many facets of education and the populations 
it impacts, it can be difficult to prioritize some areas over others or 
to decide not to fund research in some areas at all. Touch choices 
sometimes have to be made. For instance, this year, IES will not hold 
research competitions in special education.\15\ However, IES is not 
taking a haphazard approach to this dilemma. Recent discussions between 
the Board and IES staff have concerned if and how the Institute might 
decide to prioritize funding decisions. Moreover, IES is attempting to 
understand and improve the impact of the overall portfolio of research 
supported with IES funding. Together, we have been examining the 
research portfolios of NCER and NCSER to understand how IES might 
better target its research funding.
Revising ESRA: Recommendations from the NBES
    At the June 20, 2012 NBES meeting, Board members discussed specific 
recommendations to revise ESRA. These recommendations build from 
previous suggestions made by the Board in May 2008 with several 
additional changes and revisions. Most notably, we suggest:
     Establishing a requirement that the IES Director submit a 
biennial plan of activities to the Board for advice. Currently, the IES 
Director is only required to submit his or her priorities to the Board 
every six years. Although the Board has many informal opportunities to 
provide feedback to the Director based on the strong working 
relationship between the current Board and current Director, the 
expectation of more frequent formal feedback should be documented.
     Changing the term of a Board member to commence from the 
date of confirmation so that members have a full four years of service
     Automatically extending by one year the terms of Board 
members whose successors have not yet been appointed; this would help 
to ensure that the Board always has a sufficient number of members to 
be effective
     Giving the Board hiring and evaluation authority over the 
NBES Executive Director to ensure this role is independent of IES given 
the assessment duties of the Board
     Allowing for flexibility in the pay of the IES Director 
and Commissioners by making these positions eligible for ``critical 
pay'' under the Federal Workforce Flexibility Act of 2004
     Removing privacy protection for individual schools in data 
reports, a protection that does not exist in any other federal statue 
or regulation. The current prohibition on revealing school identity 
means that useful information must be omitted from evaluation reports.
    A full list of our recommendations and a marked-up copy of ESRA 
have been entered into the official record.
Conclusions
    In summary, to have an informed populace and clarity on how best to 
educate our children and ourselves, there must be a robust foundation 
of high-quality data, rigorous, objective research and strong 
communication of evidence on what works and what does not. It is clear 
that IES has made substantial contributions to our understanding of how 
to improve education and is engaged in activities to address the 
challenges it faces. There is more work to be done, and as noted by our 
recommendations, the Board believes some changes to ESRA would improve 
the functioning of IES and the Board for the continued benefit of the 
country.
                                endnotes
    \1\ Before the creation of IES, the Office of Educational Research 
and Improvement (OERI) led efforts to collect educational data. IES has 
built upon these surveys in its current activities.
    \2\ For example, the National Center for Education Research (NCER) 
serves as a hub to facilitate collaboration among a diverse, 
interdisciplinary group of researchers who are a part of the Reading 
for Understanding Initiative (RfU). NCER is funding six research teams 
to advance theories and develop interventions to improve reading 
comprehension from pre-K through grade 12. Five of the teams are 
testing interventions to improve reading comprehension through a 
variety of curricula, supplemental materials, and professional 
development opportunities.
    \3\ For instance, in September 2012, NCEE released the State and 
District Receipt of Recovery Act Funds: A Report from Charting the 
Progress of Education Reform--An Evaluation of the Recovery Act's Role, 
which documents how funding was spent and includes the characteristics 
of funded schools and districts, amounts, etc. It is part of a larger 
study of major Federal funding efforts and reflects an NCEE effort to 
get interim reports out to the public more quickly.
    \4\ Bettinger, Eric P., Bridget Terry Long, Philip Oreopoulos, and 
Lisa Sanbonmatsu. (2012) ``The Role of Application Assistance and 
Information in College Decisions: Results from the H&R Block FAFSA 
Experiment,'' Quarterly Journal of Economics, vol. 127, no. 3, pp. 
1205-1242.
    \5\ NCEE formed the technical methods group to work on issues and 
strategies that assure evaluations of education interventions provide 
unbiased and causally valid assessments. The technical methods working 
group aims to advance and provide guidance for those specialists who 
are embarking on evaluations in education. More information is 
available here: http://ies.ed.gov/ncee/tech--methods/
    \6\ For instance, in 2012, IES created the Research-Practitioner 
Partnerships in Education Research program, which supports partnering 
around issues and problems of practice identified by the state and 
local education agencies. It is administered by the National Center for 
Education Research (NCER) as part of the research grant program.
    \7\ One example of a Practice Guide is Teaching Elementary School 
Students To Be Effective Writers, which was released by the What Works 
Clearinghouse. It offers a framework and examples, and is part of 
NCEE's interest in providing practice guides that are narrowly focused 
and useful to classroom teachers.
    \8\ For example, to explain the new NAEP Technology and Engineering 
Literacy (TEL) assessment, the National Center for Education Statistics 
(NCES), created a video. The video describes what the assessment 
covers, gives examples, and makes clear that the goal of the assessment 
is to learn whether students have the skills needed to address the 
challenges of our evolving society. Additionally, an online tutorial 
allows users to get a sense of the test. More information is available 
here: http://nces.ed.gov/nationsreportcard/tel/
    \9\ Report from the National Center for Education Evaluation and 
Regional Assistance (NCEE) to the NBES concerning activities from March 
to May 2013.
    \10\ Report from the National Center for Education Evaluation and 
Regional Assistance (NCEE) to the NBES concerning activities from 
October 2012 to February 2013.
    \11\ The report entitled, Translating the Statistical 
Representation of the Effects of Education Interventions into More 
Readily Interpretable Forms, can be found here: http://ies.ed.gov/
ncser/pubs/20133000/
    \12\ More information is available here: http://ies.ed.gov/funding/
ncer--rfas/randd.asp
    \13\ Report from the National Center for Education Evaluation and 
Regional Assistance (NCEE) to the NBES concerning activities from March 
to May 2013.
    \14\ More information is available here: http://ies.ed.gov/ncee/
wwc/event.aspx?sid=28
    \15\ The National Center for Special Education Research (NCSER) 
will not hold research or research training competitions for FY 2014. 
Researchers interested in the study of children, youth, and adults with 
disabilities may be eligible for funding under the NCER competitions.
                                 ______
                                 
    Chairman Kline. Thank you very much.
    Dr. Kemple, you are recognized.

  STATEMENT OF DR. JAMES KEMPLE, EXECUTIVE DIRECTOR, RESEARCH 
    ALLIANCE FOR NEW YORK CITY SCHOOLS, NEW YORK UNIVERSITY

    Mr. Kemple. Chairman Kline and members of the committee, 
thank you for the opportunity to appear before you today.
    Continuing the theme, from my perspective as an educator 
and researcher for more than 30 years, the Education Sciences 
Reform Act and the Institute of Education Sciences have 
produced incredibly important changes in the quality, quantity, 
and use of education research.
    Until ESRA and IES, education research was allowed to 
function at a standard that would never pass muster in public 
health, employment and training, welfare policy, let alone 
agriculture and medicine. The dearth of good evidence in 
education and the inability to effectively communicate lessons 
from the little evidence that did exist left us with a legacy 
of repeatedly reinventing the wheel and chasing fads rather 
than building a reliable and useful track record of what 
worked, what did not work, for whom, and under what conditions.
    While I believe we are much better off than we were 12 
years ago prior to IES, I think we are still burdened with that 
legacy from more than two generations of ineffective research.
    From a pure numbers perspective, IES has commissioned and 
released findings from 90 studies that now meet the widely 
agreed upon gold standard for research--the randomized 
controlled trial. To my knowledge, that is 89 more such studies 
than both of IES's predecessors combined had produced, and 
there are many more in the pipeline.
    However, I believe IES's influence extends well beyond the 
scientific research studies and the activities it has supported 
over the last 12 years. The principals embedded in ESRA and in 
IES's work have also changed the way that federal, state, and 
local policymakers evaluate and use education research.
    In New York City, where I lead a partnership with the 
city's schools, the schools' chancellor, most senior staff in 
the New York City Department of Education, and, yes, even the 
mayor now ask pointed questions about whether the research that 
they are presented with meets the appropriate scientific 
standards. And I am also encouraged by the fact that when they 
do see high quality evidence they are much more inclined to use 
the resulting evidence in their decision-making even when that 
evidence suggests that programs are ineffective and should 
probably be discontinued.
    I think it is also worth noting that IES has supported 
training programs to develop a new generation of researchers 
and research organizations that are equipped to meet those 
higher standards of evidence.
    So in short, from my perspective the transformations that 
have occurred under IES have moved education research much 
closer to the caliber of research conducted for decades within 
the U.S. Departments of Education and Labor and the NIH.
    Now, while applauding IES's many important accomplishments, 
I would also describe it as a work in progress. There are 
several areas where I think the institute can be improved.
    First, in some cases I believe IES has promoted scientific 
rigor at the expense of policy and practical relevance. Second, 
IES has not invested enough in building partnerships and lines 
of communications between researchers and policymakers and 
practitioners, and those are just the people that should inform 
and benefit from the work that we produce. And then third, I 
think IES can be smarter in the strategies it uses to make its 
work accessible.
    My sense is that the original framework established by ESRA 
already provides for advancement in each of these areas, and I 
think IES is headed in the right direction. The key challenges, 
I believe, lie in helping the current leadership of IES 
continue to make strides in each of the directions I just 
outlined.
    So I want to suggest four core principles that I believe 
should guide the further strengthening of IES, and I will call 
these the four Ps: P1, preserve scientific rigor; P2, 
prioritize relevance for policymakers and practitioners; P3, 
promote wider use of high quality evidence; and P4, prepare for 
the future.
    Under P1, I think it is specifically important for IES to 
continue to place a premium on funding research that 
establishes strong, causal connections between specific 
education policies and practices and the outcomes that we care 
most about for our students: achievement in literacy, 
mathematics, and the sciences, social development, and 
preparation for college and careers.
    P2: In prioritizing rigorous research that is more relevant 
to policymakers and practitioners I think we have two 
challenges here. One is strengthening the relationships and 
collaboration between researchers, policymakers, and 
practitioners; and the second is ensuring that our studies 
address critical questions about how and why education 
practices and policies work or do not work, not just the thumbs 
up did they work or not. I think in some cases IES's pursuit of 
rigor has sometimes narrowed the scope of research to the 
thumbs up, thumbs down, did it work or not.
    And in promoting wider use and application or research, I 
think it is important that IES treat dissemination as a 
continuous process, not just an event that occurs at the end of 
a study, and that they make smarter uses of technology to 
promote and disseminate its work.
    And then finally, P4: In preparing a next generation of 
education researchers who are committed to scientific rigor, I 
believe IES should continue its support for the pre-and post-
doctoral training programs that are ensuring that our best and 
brightest are going into the field of education research.
    Let me conclude by saying this is one of the most important 
issues--education, evidence-building--that the federal 
government can play in supporting education throughout the 
United States and I urge the committee to take up the 
reauthorization of ESRA. Thank you.
    [The statement of Mr. Kemple follows:]

       Prepared Statement of James J. Kemple Executive Director,
   Research Alliance for New York City Schools at New York University

    Mr. Chairman and Members of the Committee: My name is Jim Kemple. I 
serve as the founding Executive Director of the Research Alliance for 
New York City Schools at New York University. Our organization is a 
nonpartisan research center that conducts rigorous studies on topics 
that matter to the policymakers, educators and other stakeholders who 
work with New York City's public schools. We strive to advance equity 
and excellence in education by providing evidence about policies and 
practices that promote student success. Prior to my current position, I 
worked for 18 years at MDRC, overseeing scientific evaluations of 
education, welfare-to-work, and employment and training initiatives 
across the country. Before that, I served as director for the Higher 
Achievement Program here in DC, and I taught high school math.
    Thank you for the opportunity to testify before you today. I can 
think of few more important roles for the federal government to play in 
education than its support for building and communicating rigorous 
evidence about what works to improve teaching and learning across the 
country. The current economic and fiscal environment makes it more 
important than ever to use scientific evidence to inform difficult 
decisions about how to allocate scarce resources, and to invest in 
building more and better evidence about the efforts we make to 
strengthen our schools, particularly efforts that flow from the federal 
government.
    From my perspective as an educator and researcher for more than 30 
years, the Education Sciences Reform Act (ESRA) and the Institute of 
Education Sciences (IES) have produced incredibly important changes. 
Until ESRA and the creation of IES, education research was allowed to 
function at standards that would never pass muster with public health, 
employment and training, or welfare policy, let alone with medicine or 
agriculture. As a nation, we have paid a heavy price. The paucity of 
good evidence in education, and the inability to effectively 
communicate lessons from the little scientific evidence that did exist, 
left us with a legacy of reinventing the wheel and chasing fads rather 
than building a reliable and useful track record of what worked, what 
did not work, for whom and under what circumstances. While I believe we 
are much better off now than we were 12 years ago, we are still saddled 
by that legacy.
    Since its inception, IES has funded and released findings from 90 
studies that meet the widely agreed-upon ``gold standard'' for 
research, the randomized controlled trial. That's 89 more such studies 
than all of IES's predecessors combined. However, I believe IES's 
influence extends well beyond the specific research studies and 
activities is has supported. It has changed the way federal, state, and 
local policymakers evaluate and use education research.
    In New York City, where I lead a research partnership with the city 
schools, the Mayor, the Schools Chancellor, and most senior staff in 
the Department of Education now ask pointed questions about whether the 
research that is presented to them meets scientific standards. When 
they see high-quality research, they are much more inclined to use the 
resulting evidence in their decision-making. For example, New York City 
discontinued its use of financial incentives for school performance in 
the face of solid evidence that these incentives did not improve 
student achievement. By the same token, the City has reinforced its 
commitment to creating and sustaining small schools of choice, citing 
scientific evidence that these schools are significantly improving 
graduation and college readiness rates, particularly among some of the 
city's most vulnerable students.
    IES has also helped develop a new generation of researchers and 
research organizations that are equipped to meet those high standards 
of evidence. More than 25 universities are now attracting the nation's 
best and brightest to training programs in rigorous education science.
    While these young people come from multiple disciplines, they are 
committed to conducting high-quality education research that will be 
useful to policymakers and practitioners.
    In my view, these transformations have moved education research 
much closer to the caliber of research conducted for decades through 
the U.S. Departments of Labor and Health and Human Services and the 
NIH.
    While applauding these important accomplishments, I would also 
describe IES as a work in progress. There are several areas where the 
Institute could be improved. First, in some cases, I believe IES has 
promoted scientific rigor at the expense of policy and practical 
relevance.
    Second, IES has under-invested in building partnerships and lines 
of communication between researchers and the policymakers and educators 
who should inform and benefit from its work. Third, I think IES can do 
more to make its work accessible. My sense is that the original 
framework established by ESRA allows for advancements in these areas. 
The key challenge lies in helping the current leadership of IES 
continue to make strides in each of these directions.
    I would like to organize my remarks around four core principles 
that I believe should guide the further strengthening of IES. I'll call 
these the four Ps: Preserve scientific rigor; Prioritize relevance for 
policy and practice; Promote greater use of high quality research; and 
Prepare for the future. Each of these principles should be seen as 
reinforcing and complimenting the others.
    P1: Preserve the commitment to scientific rigor. Specifically, IES 
should continue to place a premium on funding research that establishes 
strong causal connections between specific education policies and 
practices and student outcomes we care about: most notably, achievement 
in literacy, math, and the sciences; social development; and college 
and career readiness.
    Prior to ESRA and IES, the federal investment in education research 
generated reasonably good evidence about the nature of the problems we 
face in our schools, but yielded weak and unsubstantiated claims about 
how various approaches may or may not have solved those problems and 
improved teaching and learning. Even after only 12 years of work under 
IES, the education research community is finding that many of those 
claims, both positive and negative, turned out to be plain wrong.
    For example, over the past 12 years the federal government 
allocated hundreds of millions of dollars for academic enhancements to 
after-school programs, innovative teacher professional development 
programs, cutting-edge adolescent literacy programs, and computer-based 
tutoring tools. Many of these investments were guided by compelling 
theory but, due to the legacy of low quality research in education, the 
evidence base for their actual effectiveness was weak. Fortunately, 
Congress and the U.S. Department of Education had the foresight to make 
sure that these new investments were accompanied by rigorous 
evaluations under IES to learn about their impact on teaching and 
learning. Unfortunately, it turned out that most these initiatives, on 
average, had little or no impact.
    In a more encouraging example, many federal, state and local 
policymakers are currently working to expand pre-kindergarten programs. 
Due in part to the growing commitment to scientific rigor in education 
research, these policymakers no longer have to rely on one single study 
from the 1960s involving less than 120 toddlers who were exposed to an 
incredibly expensive set of services and supports before they entered 
school. Evidence from a growing number of credible studies is showing 
the benefits of affordable early intervention.
    Federal support for these kinds of rigorous impact studies is 
crucial to developing a more effective educational system.
    P2: Prioritize rigorous education research that is more relevant to 
policymakers and practitioners. This challenge has two parts: 1) 
supporting partnerships and collaboration between researchers, 
policymakers and practitioners, and 2) ensuring that studies address 
questions about how and why education practices and policies work or do 
not work.
    Prioritizing Partnerships: My organization in New York City, and 
similar groups in more than a dozen other cities around the country, 
are beginning to demonstrate the value of partnerships that include 
researchers, policymakers, administrators, and educators. By working 
together to set research priorities, interpret results and put findings 
to use, we are accelerating the pace at which research can inform 
policy and practice.
    For instance, we have worked with the New York City Department of 
Education to enhance the largest school survey in the nation, which 
collects vital information from students, parents and teachers; we have 
produced individual reports for schools involved in our studies, to 
help them improve in real time; and we have examined the effects of the 
City's high school choice process on low-achieving students, producing 
insights that have been useful to both the district and local community 
groups that are helping students navigate the system. This is a far cry 
from the typical end product of research, which generally targets 
academic colleagues and so often sits on our shelves collecting dust. 
From my perspective, IES is making strides toward promoting this kind 
of collaboration and should continue to do so.
    Similarly, the U.S. Department of Education in general, and IES in 
particular, should continue to encourage links between federal 
programmatic funding and IES-directed studies that build solid evidence 
about the impact of these investments. While IES's independence should 
remain paramount, I believe a great deal of its struggle to be relevant 
can be traced to the limited role that other offices in the Department 
of Education (as well as the Departments of Health and Human Services 
and Labor) have had in prioritizing the research questions it pursues 
and to the limited role that IES has played in getting buy-in from 
those offices.
    Until recently, IES has had even less interaction with State and 
Local Education Authorities, who in the end are the ultimate consumers 
of the evidence IES produces. This is beginning to change, with new 
initiatives like IES's Research-Practice Partnership program and the 
requirement that the Regional Education Laboratories (RELs) conduct 
their work through what are called ``research alliances,'' which are 
formal advisory groups comprised of state and local education 
policymakers and practitioners. I would encourage the continuation and 
expansion of these efforts.
    Prioritizing How and Why Questions: In my view, IES's pursuit of 
scientific rigor has sometimes narrowed the scope of research to focus 
exclusively on the question ``did it work or not?'' This obscures the 
kind of information we desperately need to make education better.
    Specifically, I would encourage IES to expand its pursuit of 
questions about how, why, for whom and under what circumstances things 
work or do not work. These questions should be essential to rigorous 
evaluations of program effectiveness and, in my opinion, will be 
especially valuable when we find something that did not work.
    For example, I helped conduct a study of what are called High 
School Career Academies, a promising school reform initiative that is 
supposed to prevent students from dropping out and help them enter the 
workforce after high school. On average, we found that the programs had 
no effect on dropout rates (although it had large positive effects on 
workforce outcomes). However, when we dug a little deeper and looked at 
students who were at the highest risk of dropping out and were enrolled 
in the most dysfunctional schools, we found quite large dropout 
reductions.
    This was in spite of the fact that the programs in these schools 
were not very strong; they were just much better than anything else 
available to those students. We were able to attribute these effects to 
the program's emphasis on personal relationships and high expectations, 
particularly in the context of an otherwise chaotic environment. We 
would not have learned this without expanding the prevue of our 
research beyond the thumbs up or thumbs down question of ``did it work 
or not.'' As a result, that study is listed in the What Works 
Clearinghouse--IES's definitive resource on scientifically validated 
research in education--as one of only seven with evidence of positive 
effects on keeping students in high school.
    Questions about how and why policies and practices work or do not 
work are important both in the context of new initiatives and when 
proven practices are being adapted to new circumstances. In particular, 
IES should continue its recent investments in what is called 
``continuous improvement research''--a process by which data collection 
and analysis are integrated into program development and 
implementation. While still in its infancy, this seems like a promising 
method for using rigorous research to help schools become more 
effective over time.
    P3: Promote wider use and application of education research. Again, 
I believe this is a dual challenge: 1) treating dissemination as a 
continuous process rather than a single event at the end of a study, 
and 2) making smarter use of technology to organize and provide access 
to high-quality evidence.
    Promoting Dissemination as a Continuous Process: In my experience, 
this also ties back to the importance of building relationships between 
researchers and the audiences we are trying to reach. For example, the 
Career Academies study I mentioned earlier was a 15-year evaluation 
(yes, 15 years). This work, involving literally hundreds of 
contributors and collaborators, has had a profound influence on career 
and technical education. While the study found that the Career 
Academies produced sustained positive effects for on long term 
workforce outcomes for young people, the story of the study's influence 
began before we collected a single piece of data. My colleagues and I 
at MDRC started this project by asking both leaders in the field and 
teachers and administrators in schools what they thought would be worth 
learning about innovative approaches to the school-to-work transition. 
We continued this dialog at each step in the study providing a wide 
range of audiences with early and long-term findings and asking for 
their guidance about how our work could be more useful. As a result, 
key stakeholders bought into the research process from its inception; 
they were able to confront the results, even though not all of them 
were positive; and, most importantly, they continue to this day to work 
diligently to reform and strengthen their programs to be better aligned 
with what we found made a difference.
    Prioritizing Smarter Use of Technology: IES has led the effort to 
bring dissemination of high-quality education research into the late 
20th Century (although probably not the 21st Century) through its 
creation of the What Works Clearinghouse--a compendium of studies that 
have been screened for scientific merit and catalogued by topic. It has 
also supported related resources like the Better Evidence Encyclopedia 
and the Society for Research on Educational
    Effectiveness. More recently, IES has issued a call for the 
establishment of a Center on Knowledge Utilization, whose mission will 
be to study how educators and policymakers use research. I believe 
these are investments worth sustaining and increasing, particularly if 
they continue their development of research-based practice guides in 
addition to their mandate to serve as arbiters of what constitutes 
scientifically valid evidence.
    However, to advance the use of rigorous research, I think IES will 
need to make smarter use of technology to make this work more 
accessible and user-friendly. I do not think this is a matter of 
keeping better track of how many reports get published or how many 
website visits they receive. This is beyond my area of expertise, but I 
am struck by the ease with which I can find pretty useful and generally 
reliable information about restaurant and movie reviews, and ratings of 
cars and appliances. I am hard pressed to believe that those of us who 
care about making high-quality research more widely available do not 
have something to learn from these efforts.
    P4: Prepare a next generation of education researchers who are 
committed to scientific rigor, to relevance for policy and practice, 
and to applying what they learn in the field. This may be the most 
important legacy of ESRA and IES.
    Hundreds of talented young people have completed or are enrolled in 
training programs supported by IES that place a special emphasis on 
teaching about scientific research methods. I have had the privilege of 
working directly with nearly a dozen of these young scholars, including 
six who are now students at NYU's Steinhardt School of Culture, 
Education and Human Development. I have been impressed by their 
competence, certainly, but mostly by their passion for making a 
difference in schools. None of these folks will see themselves as being 
successful if their primary accomplishments are to accumulate a long 
list of articles in prestigious journals and receive tenure at a 
prominent university. While many of them are certainly destined for 
these accomplishments, I am convinced that they will make their biggest 
impact in producing high-quality work that education policymakers and 
practitioners value and use.
    From that perspective, I believe IES's support for pre- and post-
doctoral training programs and for its methodological and professional 
development activities are critical investments. I particularly applaud 
IES's recent call for universities to form partnerships with schools 
and school districts as key components of their doctoral training 
programs and to ensure the graduate students spend time working closely 
with administrators and teachers to learn about their needs and 
interests.
    In closing, I think we must recognize that despite the great leap 
forward precipitated by ESRA and IES, the reality is that most states 
and school districts still use rigorous research in policy and 
administrative decisions much too infrequently. This is, in part, about 
the role that ideology and politics play (both constructively and 
disruptively), but it is also because the policy and practitioner 
communities not been very involved in the production of evidence and 
setting of research priorities. There is still very limited evidence on 
issues that matter to them; the evidence that does exist is often hard 
to understand and apply; and there is little incentive for them to 
produce or use rigorous evidence. The recommendations I am offering 
here would go a long way toward addressing these issues and would help 
make schools and school districts more active partners in education 
research.
    Of course, there are more than 14,000 jurisdictions that make 
policy and administer K-12 education in the US. The role of the federal 
government is limited at best (with only 7 percent of education 
expenditures covered by federal funding and limited capacity to manage 
implementation). It seems imperative that a not-insignificant portion 
of this limited federal investment be accompanied by two requirements--
similar to those we've seen in the Investing in Innovation Fund: 1) 
that SEAs and LEAs use federal resources to support initiatives that 
have credible evidence of their effectiveness, and 2) when such 
evidence is lacking, that they be willing to participate in rigorous 
research that will help fill this gap. Together with the four Ps I have 
proposed, over time, this approach could help our nation build a firmer 
foundation of evidence and ultimately produce better outcomes for our 
students and teachers.
                                 ______
                                 
    Mr. Rokita [presiding]. Thank you, Doctor.
    Ms. Christie, you are recognized for 5 minutes.

    STATEMENT OF KATHY CHRISTIE, VICE PRESIDENT, KNOWLEDGE/
INFORMATION MANAGEMENT AND DISSEMINATION, EDUCATION COMMISSION 
                         OF THE STATES

    Ms. Christie. The Education Commission of the States, or 
ECS, was created by states, for states nearly 50 years ago. We 
help states learn what other states are doing, what new ideas 
are emerging, and what the research says.
    We provide unbiased information. We don't advocate for 
certain education policies. And we don't pick sides.
    We are the only state-focused national organization that 
works across all levels and sectors of education, from pre-K to 
post-secondary, and across branches of government.
    The strength of the evidence underpinning policy levers and 
initiatives is critical to the success of the policy process, 
but policymakers seldom know what the evidence says on every 
issue. Much depends on the quality of their staff and whether 
they know where to go to find an evidence base. And yet, they 
have to make decisions every day, whether they can answer these 
questions or not.
    That is why having timely, succinct, and understandable 
research available is so important and why organizations like 
ECS play such a vital role.
    My role at ECS lets me sit in a national crow's nest, 
watching the horizon for the education problems states are 
struggling with and for what they are doing to solve these 
problems. But it is difficult to make sure that every education 
committee chair, governor's advisor, state superintendent, 
governing board member, and higher education entity knows about 
our resources. We very much understand the difficulties of 
getting good research into the hands of those who can do 
something about it.
    The Institute for Education Sciences has some entities 
available to attempt to address some of these needs, including 
regional education laboratories, commonly known as RELs, 
comprehensive centers, and content centers. In the past the 
production from RELs and centers seemed uneven. Resources 
seemed to take a long time to come to fruition, and by the time 
they did sometimes the window of opportunity to inform 
decisions had passed, and decision-makers were not always at 
the table to set the agenda.
    Today the What Works Clearinghouse is building a strong 
base of easily accessible program reviews and pointing toward 
interventions that work and that don't work. Summaries are now 
less academic and easier to follow. Practice guides provide 
good direction. Conclusions are presented in a more 
straightforward manner. Readers can actually easily access 
areas by topic.
    Conclusions, I am sorry. IES might consider how to more 
clearly distinguish between findings regarding whether studies 
meet standards of evidence and whether programs actually impact 
learning. Overall, however, the site has improved greatly. We 
link to the clearinghouse by issue area, which is an efficient 
way to immediately capture updates for our constituents.
    In addition, content centers on issues of importance to 
states, such as turnarounds and state capacity, can be spot on 
for meeting state needs. But IES could work to ensure that 
vetting and review for activities and outputs does not inhibit 
the development of timely, relevant, digestible research and 
assistance.
    The Best Evidence Encyclopedia, which has funding ties to 
IES, is another excellent resource. And my understanding of the 
new breed of RELs is that they are working to establish 
research alliances across those states with state leaders at 
the table.
    To put these entities in perspective, though, I would like 
to highlight what is good about one of my favorite sources, the 
National Bureau for Economic Research. Nearly every week I get 
a message highlighting several of their new studies.
    These studies are relevant to the problems I see states 
struggling with, such as compulsory attendance. They are 
timely; they are prolific. IES could look to this model for 
improving relevance and timeliness.
    Like most academic studies, these are so academic that 
people are not going to read them, and for the most part they 
are not openly accessible. So ECS is working to translate 
studies like this and capture the key findings and 
recommendations in our research studies database. We organize 
them by frequently asked questions such as, ``Preschool: How 
prepared do teachers need to be?''
    Since 2008 we have entered 193. We are very thankful for 
the GE Foundation for supporting this work.
    The reason for this effort is clear: When busy people ask, 
``What does the research say?'' any response to that question 
needs to be timely, relevant, digestible, and trusted.
    Here are the four final points I would like to make.
    One: Research matters not only to those implementers in the 
field--the superintendents, principals, and teachers--but to 
those who are committed to improving the system of education.
    Two: The gold standard matters, but while optimal, it is 
not always possible. The real world will continue to demand 
that policies be crafted based on hypotheses that are 
relatively well supported by evidence or whether early evidence 
is simply promising. IES could do a better job of ensuring that 
topics fit with what matters to states and that its research 
helps answer not only which programs work but also which 
policies or state investments hold promise and which elements 
of those policies matter most so that state-level elected 
officials might act on them.
    Three: IES needs state leaders to perceive it as an 
unbiased, honest broker, so increasing the independence of IES 
could be key.
    And four: IES should consider a coordinated effort to 
transparently evaluate and hold itself accountable on a set of 
performance measures that are important to states.
    Thank you for letting me be here.
    [The statement of Ms. Christie follows:]

    Prepared Statement of Kathy Christie, Vice President, Knowledge/
  Information Management & Dissemination, Education Commission of the 
                                 States

    Mr. Chairman and Members of the Committee: My name is Kathy 
Christie. I am Vice President, Knowledge/Information Management & 
Dissemination for the Education Commission of the States (ECS). The 
Education Commission of the States (ECS) was created by states, for 
states, almost 50 years ago.
    Since 1965, ECS has worked with state policymakers to improve 
America's public P-20 education system. We provide unbiased 
knowledgeable advice to state leaders and help them to learn from one 
another's experiences. What makes ECS unique in the crowded education 
policy arena is that we work with policymakers, researchers, thought 
leaders and practitioners across all levels and sectors of education--
from pre-K to postsecondary--and across branches of government. We are 
the only state-focused national organization that brings together 
governors, state legislators, chief state school officers, higher 
education officials, and business leaders to advance policies that 
improve our educational system. To accomplish this work, we undertake 
the following kinds of activities:
     We help states learn what other states are doing, identify 
best practices, what new ideas in education are emerging, and what the 
research says.
     We provide unbiased information. We don't advocate for 
certain education policies and we don't pick sides.
     Our website is one of the best in the country to find 
information on hundreds of education issues. You can check out our 
website or call us directly; either way, we will provide the 
information that's needed.
    Most people are not aware that it was ECS--the only nationwide 
interstate compact for education--which was responsible for the 
creation of NAEP, the National Assessment of Educational Progress. It 
is this kind of state collaboration and state insight that makes ECS 
unique. The message I bring here today is about the power of state-
level leaders to make a difference in the educational opportunities 
available to children.
State Leaders Influence System-Level Reforms
    Policymakers at every level in states--chief state school officers, 
governing board members, legislators, and governors--all have a role to 
play in developing and implementing education initiatives. Very often 
these various policy actors have different opinions about how laws 
should be shaped. Consequently, evidence related to the hypotheses that 
underpin state-level policies and evidence on the relative 
effectiveness of state-level initiatives are critical to the success of 
the policy process.
    They hope, and we do too, that the decisions they make will drive 
improvements in what can be a large, bureaucratic system. A constituent 
of ours once described the education system as a big pillow. You punch 
it--and make a big dent, but gradually that dent disappears and the 
pillow puffs out to its original shape. You punch it again, making 
another dent. But after a bit the pillow again puffs back out to its 
original shape. We all understand how tough it is to make real change.
    State-level policymakers seldom know what the evidence says on 
every issue. Much depends on the quality of their staff and whether 
they know where to go to find an evidence base. Even when they have 
capable staff with sufficient time, it can be a challenge to gather and 
prepare evidence in a way that allows policymakers to quickly 
understand whether it is sufficiently robust to support the decision 
they will make.
    When considering a hypothesis behind a bill, legislators routinely 
ask: What is the level of evidence supporting this proposal? Is it 
minimal, with a higher risk to success? Is there some evidence 
available, but it fails to highlight the most critical factors that 
need to be in the policy? Or is there extensive preliminary research 
and piloting, with interventions that have been aligned at all levels 
and across agencies--a sufficiently robust knowledge base on which to 
guide large-scale decisions?
    And yet state legislators have to make decisions every day--whether 
they can answer these questions or not. That's why having timely, 
succinct and understandable research available is so important and why 
organizations like ECS play a vital role in state-level education 
policy.
National Perspective
    My role at the Education Commission of the States lets me sit in a 
national crows' nest, watching the horizon--across state boundaries--
for the education problems states are struggling with and for what they 
are doing to solve those problems. ECS scans news clippings every day 
and pushes the most relevant out via email--every day. We track the 
policies that state legislators are enacting, and we add them every 
week to the most extensive, freely-available database of its kind in 
the country. Every day we are culling from the professional and 
academic literature and pushing the best back out via our web site. But 
it is difficult to make sure that every chair of an education 
committee, every governor's education policy advisor, every state 
superintendent, every state board member and every higher education 
agency head knows about those resources. We very much understand the 
difficulties of getting good research into the hands of those who can 
do something about it. And these challenges don't even begin to touch 
the difficulty of reaching every legislator and agency head and 
governing board member across the country.
    What I have learned in over 20 years with ECS is that we reinvent 
the wheel time and time again. Policymakers don't pay enough attention 
to history. We might read the research and go, ``oh, yeah, that's 
right'' but then we too often jump at some new shiny, glittery answer 
or lobby for a new research study rather than taking time to unearth 
the root cause of a problem or step back to analyze the existing 
research--the research that while there, might not be broadly available 
or is so incomprehensible, we don't know what to make of it.
    The Institute of Education Sciences has some entities available to 
attempt to address some of these needs, including Regional Education 
Laboratories (commonly known as RELs), comprehensive centers, and 
content centers. Past RELS seemed uneven in production of resources, 
particularly those that might remain relevant and useful long after 
individual instances of technical assistance or convenings. Resource 
development or projects seemed to take a long time to come to fruition, 
and by the time they did, sometimes the window of opportunity to inform 
decisions had passed. And decision-makers were not always at the table 
to set the agenda.
    Today the What Works Clearinghouse (WWC) is building a strong base 
of accessible program reviews and pointing toward interventions that 
work--and that don't work. WWC has improved over time. Summaries are 
now less academic and easier to follow. Practice guides provide good 
direction for practitioners. Conclusions are presented in a more 
straightforward manner. Readers can easily access areas related to 
specific topics. IES might consider how to more clearly distinguish 
between 1) findings regarding whether studies meet standards of 
evidence and 2) evaluations of actual program effects on learning. 
Overall, however, the site has improved greatly. ECS is able to link to 
WWC issue areas via relevant topic areas (e.g., literacy) on 
www.ecs.org, so as studies are added to WWC, it is not necessary for us 
to add each new review to our site. This is efficient and immediately 
captures updates for our constituents.
    Content centers that focus on topics that matter to states--
turnarounds and state capacity, for example--can be spot on for meeting 
state needs. IES might review processes to ensure that vetting and 
review processes for activities and outputs of these new centers and 
for the new RELs does not inhibit the development of timely, relevant, 
digestible research and assistance.
    The Best Evidence Encyclopedia is another excellent resource: a 
free web site created by the Johns Hopkins University School of 
Education's Center for Data-Driven Reform in Education (CDDRE) under 
funding from the Institute of Education Sciences. And my understanding 
of the new breed of Regional Education Labs (RELS) is that they are 
working to determine what states need and helping to establish research 
alliances across those states--with state leaders at the table. But 
like many education policy organizations, including ECS, RELs may 
struggle to raise awareness of available resources and how they can 
support their states and school districts.
    From my crows' nest, I seek out and cull research reports from the 
comprehensive centers and the content centers--although some are more 
prolific than others.
    To put this comment in perspective, I'd like to highlight what's 
good about one of my favorite entities--the National Bureau for 
Economic Research. Every week I get an email summarizing the education 
studies they have completed. Every week. Not four per year. Not one per 
month. Every week. They're called Working Papers, so one's immediate 
assumption is that they are food for thought--not to be considered 
definitive, but findings to think about. Why are they so compelling? 
They are relevant to the problems I see states struggling with--like 
compulsory attendance, for instance. Does the age at which kids start 
school matter? Does the cut-off age (5 by September 1, for example) for 
attendance matter? Does the upper compulsory attendance age make a 
difference over the course of a lifetime? NBER studies are relevant and 
timely. They look at the types of issues that governors and state 
legislatures can influence via policy. They are prolific producers of 
what most would agree is quality research.
    These studies are dense. Their titles are often just abysmal and if 
they were movies, no one would buy tickets (e.g. Interpreting the 
Evidence on Life Cycle Skill Formation), even though they have much to 
offer. They are so academic that most state policymakers--or their 
excellent staff--are not going to read them. The same goes for studies 
coming out of AERA. And for the most part, the studies are not openly 
accessible. But they have something to say to you. And to state 
legislators. And to governors. So ECS is working to translate studies 
like this and capture the key findings, recommendations, and 
implications for policy in its Research Studies Database. We organize 
them by frequently asked questions such as ``Preschool: How prepared do 
teachers need to be?'' or ``High school curriculum: How important is 
rigor?'' Since 2008, we have entered key findings and policy 
implications from 193 studies into The ECS Research Studies Database. 
We are very thankful to the GE Foundation for supporting this work.
    The database could be easier to use. It could be ``prettier.'' We 
have created standards for inclusion, and an important standard is that 
studies need to have appeared in peer-reviewed journals. But we do make 
exceptions.
    For example, I personally mined Crossing the Finish Line--a 2009 
book by William Bowen, Matthew Chingos and Michael McPherson--names 
probably familiar to many in this room--that looked at what impacts 
student persistence and success across postsecondary institutions. 
Compelling, compelling statements built from statistical analyses are 
buried throughout, but they were most certainly read by more academics 
and higher education leaders than by policymakers. The authors' 
analysis supports, for example, the clearly articulated statement that 
``both parental education and family income are strongly associated 
with graduation rates even after controlling for related differences in 
student characteristics, particularly academic preparation. However, 
family income, not parental education, is primarily responsible for the 
overall relationship between SES and time-to-degree.''
    If another credible, vetted resource emerges to counter such 
findings, ECS will not hesitate to include it in the database as well.
    As exceptions to the peer-review rule, we include NBER studies, 
many of which eventually are published in peer-reviewed journals--but 
in the interim, they reflect the food for thought that state leaders 
need.
    Do enough state leaders know about this resource? No. Have we been 
successful in marketing it? Probably not. We always include new studies 
in our weekly e-newsletter, e-Connection, and they always get the most 
hits; regrettably, we have not been particularly successful in 
marketing its availability.
    The reason we acted, though, is clear. When a consistent element of 
questions is ``what does the research say?'' the response needs to be 
timely, relevant, digestible, and trusted.
    Here are the four final points I would like to make:
    1. Research matters not only to those implementers in the field--
the superintendents, principals and teachers--but to those who are 
committed to improving the system of education.
    2. The gold standard matters. But while optimal, it is not always 
possible. The real world will continue to demand that policies be 
crafted based on hypotheses that are ``relatively well'' supported by 
evidence or where the early evidence is ``promising.'' IES could do a 
better job of ensuring 1) that topics fit with what matters in states; 
2) that its research helps answer not only ``which programs work'' but 
also which policies or state investments hold promise--and which 
elements of those policies matter most so that elected officials might 
act on them.
    3. IES needs state leaders to perceive it as an unbiased, honest 
broker, so increasing the independence of IES could be key.
    4. In that regard, IES might want to consider a coordinated effort 
to transparently evaluate and hold itself accountable on a set of 
performance measures that are important to states.
                                 ______
                                 
    Mr. Rokita. Thank you, Ms. Christie.
    Thank the witnesses for their testimony.
    I now would like to recognize Chairman Walden, Walberg for 
5 minutes.
    Mr. Walberg. Thank you, Mr. Chairman. Put a W in front of 
it and we are always at the end, so we all look the same. Thank 
you.
    Appreciate the panel being here today, and in the process 
of doing a lot of reauthorization of laws that are in place, 
agencies that are in place, the question continues to come to 
my mind of why.
    Ms. Christie, should reauthorization of Education Science 
Reform Act take place?
    Ms. Christie. I think IES is an incredible resource. Like 
all of us, it can get a lot better.
    Mr. Walberg. Mr. Scott, I would ask you the same question. 
From your perspective with GAO, should reauthorization of the 
Education Science Reform Act--of course that is very pertinent 
to IES--take place?
    Mr. Scott. Well, I think as our work demonstrates, IES has 
played a significant role in improving the quality and the 
rigor of education research, and given the number of efforts 
that are currently underway to reform the U.S.'s educational 
system, it is critically important that we continue to have 
some vehicle such as IES to ensure that the various programs 
that are underway can be properly evaluated.
    Mr. Walberg. Ms. Christie, is it the responsibility of the 
federal government to do this research, or could it be done 
equally as well--maybe more efficiently--at the private sector 
level and, in fact, at the states in competition for 
educational quality, promoting research entities that would 
definitely be looking at states and their responsibility for 
education?
    Ms. Christie. We love to highlight states that in every 
bill put in an evaluation component to--so that they are 
modeling continuous improvement to their constituents, their 
schools, their districts. Many times it is the cost component 
of that that is difficult, and there are other times when if 
there are common problems across the states--and there are; 
they are nearly all common even though we all would like to 
think we are absolutely unique--but when you have common 
problems that is the power of ECS. You do things as a 
collaborative. You do things that keep the scale within reach.
    And I think the federal level has always had a role in 
keeping the spotlight on equity and ensuring all kids have 
opportunities, and I think that there are certain things that 
are probably fitting for both to do.
    Mr. Walberg. I guess my concern comes back that also on the 
federal level bureaucracies develop that take $600 million of 
resources that certainly could be effective at a private sector 
entity, looking for best practices with those unique students 
we have. And yet, as you I think accurately state, with their 
uniqueness there is a great deal of sameness as well, that 
parents, local school boards, school districts, 
superintendents, teachers want to achieve in the outcome of 
students.
    And I guess I am not hearing, as of yet, a strength in the 
answer that this could not be done in somewhat a market-based 
approach at the local level--of states specifically--of doing 
the research that is necessary with best practices that are out 
there and a clear understanding that we need to achieve those 
and how do we do that.
    So respond to that, Ms. Christie, if you would, please?
    Ms. Christie. The big component here is trust. So if you 
have a lot of private sector folks doing research, which even 
right now, I mean, when IES contracts with REL providers, those 
are basically private contractors.
    There needs to be a sense that what is coming out is 
unbiased and that it can be trusted. And I worry that if that 
gets outside of an independently verified group that you could 
lose that trust, and I think that would be a huge gap, then, in 
the research.
    Mr. Walberg. Well, Mr. Chairman, I would yield back my time 
after saying that if we are talking about trust in context with 
the federal government and independence in context with the 
federal government we have a major hurdle to get over.
    I yield back.
    Mr. Rokita. I thank the gentleman.
    Ranking Member Miller is recognized for 5 minutes.
    Mr. Miller. Thank you very much.
    Sounds like there is a considerable amount of trust here at 
this table in this process, with some suggestions for 
improvement--but not eradicating the concept that IES has been 
put here.
    Dr. Kemple, you--I think the panel seems to agree that we 
are doing pretty well on the rigor side here. The questions is 
about how do you get that dissemination to take place that is 
actually useful at the state level or even at the district 
school level.
    And that has always been a problem for me. I would say 
before we got into this process, what we had was a landscape 
littered with 5-year pilot projects that just died and nobody 
ever asked what happened to them or did it work or not, and 
then somebody came along over and over and over again, very 
often with the same pilot project just in a different state, 
different district around here.
    So how are we doing on this dissemination in terms of 
following through on it--not just getting it there, but then 
what is done to help people if the state decides they want to 
go through it? I mean, we are sort of going through this with 
Common Core. Our governor just put up $1 billion to help the 
districts with the implementation. I mean, this is a big hurdle 
in terms of taking new findings of rigorous research and 
suggesting to people, ``This may be a better way to do 
something than you have been doing for 15 years the other 
way.''
    How are we doing on that hurdle?
    Mr. Kemple. I think this is a two-fold problem. I think it 
starts with thinking about dissemination and promoting the use 
of research as a process rather than as an event. Too often we 
place much too much emphasis on waiting until the study is done 
until we actually think about what we have been able learn from 
that.
    I think, A, it is important to think about dissemination as 
it is starting before even one iota of data is collected to 
make sure we are asking the right questions; I think it is 
important that we incorporate the policymakers and the 
expertise and the practitioners who are part of the formation 
of the research projects or the evidence-building process in a 
continuous way throughout a project so that we are feeding the 
information that we are learning in process into a decision-
making process and into a learning process.
    And I think thirdly, it is important to make sure that as 
we package results, that as we put results together, we are 
translating those into practice guides so that we are being led 
by the best evidence possible in a way that is actionable and 
that makes the practitioner community--or puts the practitioner 
community in a position to learn from both the work in progress 
and the work at its conclusion.
    Mr. Miller. Dr. Long, this is your field. How is it going?
    Ms. Long. I would say that IES is doing a wonderful job in 
terms of production and leveraging the field to produce the 
research. I think the dissemination part of it is a work in 
progress.
    And actually, as the board and IES have talked about it 
much more in terms of communication, which is a two-way street 
between IES, the work that is produced, and what is known with 
the field and what the field needs in order to do better work.
    Part of what is involved with that is translation--taking 
very rigorous research, which education has gone forward 
tremendously in having causal results, but translating that for 
the layperson, for the teacher who is in the classroom so they 
know how to use it, and then learning how to disseminate it.
    IES needs to do more----
    Mr. Miller. Ms. Christie, is that your job? At that stage 
is that your job, the handoff here to the states?
    Ms. Christie. We hand off everything we can get our hands 
on to the states, yes----
    Mr. Miller. No, but I am just asking, you know, you get 
this new research, you get a model for dissemination or to the 
importance of this research, and I just want to--so where do 
the states pick this up and make a decision?
    Ms. Christie. Yes, we----
    Mr. Miller. Not all research is welcome, you know, I find 
from time to time in the education establishment because it 
suggests substantial change--significant change. So how do the 
governors pave the way when you have rigorous research that 
suggests you have got to change directions? I mean, we are now, 
what, 2 years into the Common Core with governors modifying and 
rethinking the model back and forth to where we are today.
    Ms. Christie. Well, if I could make one suggestion, so much 
of what is in the What Works Clearinghouse, for example, is 
about the programs, not necessarily about those big policy 
issues that a governor needs to take up.
    But I think research is welcome, it just needs to be 
crafted in a way that folks like me--I mean, I am just a 
translator; I am not an academic--that I can understand and 
then I can put into practice.
    Mr. Miller. Well, Mr. Kemple, how do we get the feedback 
loop back to the researchers and the disseminators at IES based 
upon what you have learned in modification? Because if you look 
at teacher sites, teachers are always modifying somebody else's 
lesson plans, somebody else's approach, and it is rather an 
interesting process. How do you get what you are doing in New 
York back to the IES and others about how this is going?----
    Mr. Kemple. Well, I think just in the way that a teacher 
collaborative, as you mentioned, works, as someone is learning 
from--taking what one teacher has done and they are building 
from another, those folks are all at the table together talking 
both about what their goals and objectives are, what kinds of 
problems they are facing in the classroom, and what kinds of 
different solutions they have each come up with on their own. 
Again, I think this is a process of making sure that all of 
those folks are somehow at the table as we discuss the nature 
of the problems that we face, the solutions we have each come 
up with, and then cycling that back into an effort at 
continuous improvement.
    Mr. Miller. Thank you.
    Mr. Rokita. I thank the ranking member.
    Mr. Tierney is recognized for 5 minutes.
    Mr. Tierney. Thank you.
    Thank the witnesses for their testimony. This, I think, 
follows along with Mr. Miller's questions a little bit, and it 
may--I think it is a fair question to ask. Can any of you tell 
me about a program or practice that was actually affected by 
the research--one that was either scaled up or eliminated or 
changed in some way?
    Mr. Kemple. Sure. I was privileged to lead a study of what 
are called high school career academies, which are high school 
reform initiatives aimed at dropout prevention and helping 
young people make healthy transitions into the workforce and to 
college. This is a 15-year study--yes, 15 years. So if we had 
waited 15 years for this to translate into some change in 
practice I think we would probably have lost any momentum or 
chance of making a difference.
    But this was an initiative that found little or no impact 
on college readiness, although many of the young people in the 
study went on to college, but found massive effects on their 
capacity to find good jobs, keep good jobs, and climb a career 
ladder in the 8 years following high school graduation. We were 
able to form a coalition of stakeholders in school districts, 
at the state level, and among expert organizations across the 
country that were part of helping us form that research project 
and part of the process that we used to disseminate what we 
were learning over the course of the 15-year study, so that by 
the time we got toward the end of that project many of those 
groups had already begun to synthesize the work into the 
creation of organizations that were aimed at supporting the 
standards for implementing--for creating and sustaining these 
career academy programs and for a continuous improvement 
process to work on some of the weaker aspects, such as the 
academic curriculum and the college access programs.
    Mr. Tierney. So 15 years--that is before the 2002----
    Mr. Kemple. That is right.
    Mr. Tierney [continuing]. Authorization of this act. So was 
that done under this act or something else?
    Mr. Kemple. That was actually done independently through 
private funding.
    Mr. Tierney. Okay. Well, my question to you is can you give 
me an example of something that occurred under this act that 
either changed in some way a program or a practice----
    Ms. Christie. Dr. Long?
    Ms. Long. Yes. I can speak about some of my own work that 
was supported by IES.
    Mr. Tierney. Please.
    Ms. Long. Working with two of my colleagues, Eric Bettinger 
and Phil Oreopoulos, we worked with tax preparers to help 
families fill out their student financial aid forms, the FAFSA. 
And so after completing their taxes we only needed 5 additional 
minutes to pre-populate the FAFSA using software and then 
asking them a few additional questions as well as giving them 
information about college prices and what they needed to do in 
order to access those institutions. We found huge results in 
this randomized controlled design of getting many more students 
into college by just offering 5 minutes of assistance.
    This report was first released in 2009, right around the 
debates about FAFSA simplification, was it worthwhile, and we 
have subsequently gotten an additional grant from IES to 
continue to see how we can expand these kinds of services and 
community tax sites around the country.
    Mr. Tierney. And did the FAFSA reforms in any way play off 
of those findings, and did they----
    Ms. Long. We certainly fed that information into the 
debates and received many calls from both states as well as 
many members of Congress around the federal government to 
understand that, yes, information is a huge barrier. This was a 
randomized controlled trial, simplifying things and giving 
assistance, and it turned out it was very cost effective.
    Mr. Tierney. Dr. Scott, or Mr. Scott, do you find enough of 
those examples in your work, in your review of this agency to 
continue to warrant its continuation?
    Mr. Scott. I think, as we noted in my statement, IES is 
uniquely positioned to act in this area. I think it is 
important, though, that as we also pointed out, that they take 
some steps to improve. For example, in the areas of continuing 
to get feedback from practitioners and policymakers, continuing 
to hold various aspects of their operations accountable, such 
as the RELs, and then continuing to measure and report out on 
their activities. So I think IES is uniquely positioned to 
contribute in this area but there also needs to be some 
improvements to their operations and accountability.
    Mr. Tierney. Thank you.
    Do practitioners--do actual teachers and superintendents 
and principals--have a way of connecting with IES and looking 
at these reviews periodically and interacting? Is there some 
setup to encourage that?
    Ms. Long. There are several efforts. There is information 
that is available online. There are constant efforts to 
translate and make this information available. For example, the 
practice guides have been cited, which really break down the 
research to what teachers can do in their classrooms.
    But this is a work in progress. Much more needs to happen, 
and IES is taking steps to get more feedback from the field 
about what they need and how they need to use that information.
    Mr. Tierney. And are you working with colleges that teach 
people to become teachers as to how they might access a tool 
like this and make it part of their overall practice as they go 
out and teach?
    Ms. Long. That is one of the audiences, but much more could 
be done with them.
    Mr. Tierney. Thank you.
    Thank you, Mr. Chairman.
    Mr. Rokita. I thank the gentleman.
    Ms. Bonamici is recognized for 5 minutes.
    Ms. Bonamici. Thank you very much, Mr. Chairman.
    And thank you to the witnesses for being here today for 
this discussion. I, especially as a member of this committee, 
truly believe that education is an investment in our country's 
future and so it is important for us to get the policy right 
here.
    And, Ms. Christie, I want to say that as a former state 
legislator, thank you. Frequently relied on the Education 
Commission of the States and your work there.
    I especially appreciate your pointing out in your written 
testimony that you have learned over the years that we tend to 
reinvent the wheel, and also pointing out that it is important 
to take the time to really unearth the root cause of a problem, 
and I hope that this hearing today will help with both of those 
areas--help us not reinvent the wheel, but also not neglect the 
importance of looking at the root cause of the problem. And I 
have often spoken about the importance of, for example, looking 
at poverty and homelessness and how that affects students in 
school.
    So I wanted to talk a little bit about what many of you 
have mentioned both in your written and oral testimony, and 
that is the relevance of research. So I am from Oregon, and we 
have the Oregon Leadership Network Research Alliance, which is 
under our Education Northwest, and they have really been 
working to bring together the practitioners on the ground with 
the policymakers, understanding that the research needs to be 
accessible to both of these groups, and I agree.
    So I would like to talk a little bit about what relevance 
means.
    And, Dr. Kemple, you said that many states and school 
districts don't use rigorous evidence because they have not 
been involved in producing or guiding what research is done. So 
you also mentioned that it is important to prioritize 
educational research that is relevant to policymakers and 
practitioners.
    So can you all talk a little bit about what does relevant 
mean and who is determining relevancy? So are the practitioners 
and the policymakers all saying that the same things are 
relevant? And how do their views differ?
    And then who ultimately makes that decision? If we are 
saying we want relevant research, who is going to make that 
decision? Is that going to be us in--as policymakers, is it 
going to be the practitioners, or is it going to be the 
researchers, and why?
    Mr. Kemple. So briefly, I think relevance means two things. 
One, I think it means, whose questions are we asking as 
researchers or how do we--whose questions are we prioritizing 
as researchers? I think that is a two-way conversation. I think 
it has to include people who are the ultimate consumers of 
their research. If it is a question for Congress about whether 
they would like to make sure that an investment in early 
childhood education, adolescent literacy, afterschool 
programming is paying off in terms of improving--achieving the 
goals that it was set out to, either improving teaching or 
improving learning among young people, we ought to know that 
those are priorities for what gets learned and that 
conversation needs to happen between the people who are making 
the tough decisions about how to allocate scarce resources for 
the benefit of children.
    Secondly, though, I think it means that we have to place a 
higher priority on the questions about why and how and under 
what conditions are investments in education improvement make a 
difference or don't make a difference. In my view, it is just 
as important to find out something doesn't work as it is to 
find out that it doesn't work--or that it does work, and--but 
going beyond the thumbs up, thumbs down to say, ``Why did this 
work? What were the causal mechanisms? What was the appropriate 
context?'' Those are the things that will matter most and 
translate well enough in Oregon as well as New York State or 
Arizona or Texas.
    Ms. Bonamici. Thank you.
    And from the other witnesses, do you want to give me some 
input on how those determinations are made about what research 
is done? How is it determined that it is relevant? And how can 
we better improve those--that communication between the 
practitioners and the policymakers and the researchers?
    Ms. Long. I think the importance of partnerships cannot be 
underscored enough. And this is a movement that IES is going 
towards, that there has to be a feedback mechanism where you 
are doing research in partnership with the field, with the 
teachers, the schools, the universities, so that as you learn 
as you go along and it is a continual improvement type of 
model. Because what is relevant can also change very quickly.
    At the same time, you need a federal organization that is 
going to be objective and look at the national good and say, 
``This is what is important for the country. This is the 
information that we all need to know.'' And some of the myths 
that we have had before, we need to realize those don't work 
and other things do, while also feeding that information to 
people who are making decisions--the policymakers--to 
understand, given our limited resources, we have to make 
decisions, we have to prioritize. So that is also very 
important and relevant.
    Ms. Bonamici. Thank you.
    Ms. Christie?
    Ms. Christie. Sure. I would like to push on a little bit 
different way that might not have been talked about in the 
past.
    There are groups like ECS and the National Conference of 
State Legislatures, National Governors Association, Council of 
Chief State School Officers, the state boards organization, 
that if someone reached out to all of us we could weigh in on 
what we hear from the states. I mean, I watch every day e-clips 
across the country, we have a policy database that even someone 
at ECS could put together a team and sit down and actually look 
at what are the policies that are being enacted and what are 
states struggling with? Because they are like the tip of the 
iceberg for what people need to know.
    Ms. Bonamici. Thank you.
    I see my time is expired. Thank you, Mr. Chairman.
    Mr. Rokita. Thank the gentlelady.
    Ms. Foxx is recognized for 5 minutes.
    Ms. Foxx. Thank you, Mr. Chairman.
    And I want to thank our panel for being here today.
    And I want to say, I want to associate myself with the 
train of questioning that Mr. Miller was going at before, I 
believe, and other folks, in terms of the way we are looking at 
this issue, and particularly in the way of application. That 
old saying about deja vu all over again, I have been in this 
business for a long time and this panel--we could be in a time 
warp here, actually, because I believe I first heard this kind 
of debate and this kind of discussion back in the 1960s and 
maybe 1970s and 1980s when I was on the school board, when I 
did my master's degree.
    We have been dealing with this issue forever in terms of 
how do we get the research--how do we get the knowledge that we 
have applied appropriately in the places where it can do the 
most good? And it seems as though we haven't figured that out 
yet.
    I mean, people have been decrying the fact that folks in 
education just ignore the research and the results.
    So what I would like to ask each one of you is do you have 
some examples of where you have seen the research that we--of 
programs that work well, methods that work well--where has it 
been applied well, appropriately, and how do we replicate those 
situations without it costing a lot of money?
    Anybody?
    Mr. Kemple. I guess one example I would point to for now 
is--and highlight the fact that I think one thing that really 
does differentiate us now from the 1960s, 1970s, 1980s, and 
even 1990s is that what was not on the table then was whether 
we had evidence we could really believe. I think by and large 
we were making big investments in research that really didn't 
provide us with any causal link between the programs and 
policies that were being paid for and the outcomes that we 
cared about for the young people in the schools.
    I think one--what we are starting to see--and among those 
things were teacher preparation. I think what we see is a long 
litany of research that suggests that it doesn't matter how 
much--where people were trained, what their test scores were 
coming out of graduate schools of education, and whether, in 
fact, they had a master's degree. What mattered most was their 
experience in the classroom and the way that they were 
mastering their subject matter material.
    We now have some pretty solid evidence that some 
alternative routes to certification or entry into teaching are 
turning out to be as if not more effective than traditional 
routes through schools of education. I think we are starting to 
see a proliferation of strategies to draw in talented people 
from across the country to opt for teaching as opposed to other 
highly prestigious occupations, and some of that is growing out 
of the research that suggests that initiatives like Teach for 
America or the New Teacher Project are producing positive gains 
for young people in classrooms.
    Ms. Long. To build from that, even today there is the 
release of a report looking at alternative certification 
programs that is being released that was partly funded by IES, 
and it has completely changed the way people discuss what is a 
high quality teacher. Many of the old models that we have been 
using for years of just get a master's degree have been called 
into question and because of the data that IES collects we know 
that the way that teachers are distributed across schools 
varies incredibly. Where there are high-need schools with 
students that are suffering, we have to figure out policies of 
how to get teachers there.
    I think the other thing IES has definitely changed is the 
way that people are trained to do education research and who is 
attracted to do education research. You now see in graduate 
schools of education people who used to be former teachers, who 
worked in community-based organizations, who now, combined with 
all of these rigorous tools, are doing research in a very 
different way than they did in the past.
    The other thing that is very different now is technology, 
and this is something that IES is grappling with: How can you 
use technology to communicate and get feedback from the field? 
And that is everything from building a Web site, which is 
certainly not enough, to podcasts and other kinds of uses of 
technology, and we have seen several examples--everything from 
how they are communicating with teachers to parents to trying 
to distill some of this information so that they can use it.
    Ms. Christie. Can I weigh in?
    The early literacy and the early learning research I think 
has had a dramatic effect. The problem is I graduated with a 
teaching degree in 1970, and at that time I knew that the first 
three years of a child's life were the most important ones, and 
that is why later in my career I decided to stay home with my 
kids for 12 years.
    So you are absolutely right about the recycle. And ECS had 
a huge early learning initiative in the 1970s, so we have 
known. The trick is, exactly, why do we keep moving on to the 
next easier thing to do than getting back to what we need to 
do, how do we push that back out, how do we make sure we are 
actually making progress? And those are the really tough 
questions, I believe.
    Mr. Rokita. I thank the gentlelady.
    Ms. Foxx. Can I indulge for 10 seconds?
    Anybody who comes in this room who is in the education 
business and uses the word training will get my lecture. We 
train animals and we educate people. So those of you who are in 
the business of education, I would ask you to not talk about 
training but talk about education.
    Mr. Rokita. I thank the gentlelady.
    Mr. Scott is recognized for 5 minutes.
    Mr. Scott of Virginia. Thank you, Mr. Chairman.
    And I have some of the same questions that the ranking 
member had about what you actually do with the information. For 
example, I notice that the What Works Clearinghouse studied 
many dropout prevention programs. Exactly what do we know about 
dropping out and what can be done about it on a local level?
    Ms. Christie. Can I jump in?
    Mr. Scott of Virginia. Sure.
    Ms. Christie. I think the data that is some of what IES is 
funding--for example, with early warning indicators and that 
sort of thing--is a tremendous leap forward, and I think states 
are paying a great deal of attention to that, trying to figure 
out how to even get those early warning indicators down into 
the very early grades to identify kids very early on.
    The interventions that are in the What Works Clearinghouse 
on trying to prevent kids from dropping out of school, I think 
there is the kind of information there that people can get 
their arms around.
    I have to tell you that at the state level we moved offices 
several years ago and we had box after box after box of state 
dropout initiatives. There was not one piece of evidence tied 
or--in any of those boxes. Nothing had been evaluated.
    So it is just so critically important, and it doesn't 
always get done.
    Ms. Long. I think the goals of the What Works Clearinghouse 
are very admirable. Education is a very crowded space and there 
are many opinions and there are many organizations that are 
releasing different reports.
    And because much of the research is very complex, the 
average person oftentimes cannot understand it and so IES has 
stepped in as a federal role to say, ``We will be an objective, 
unbiased source. We will help to translate what all this 
complex work means.''
    And that is where the What Works Clearinghouse steps in, 
looking at studies to say, ``People are making these grand 
statements. Can you actually believe these results?''
    To walk away and say, ``This program does or does not 
work.'' Knowing that it does not work is also very important.
    But the What Works Clearinghouse, I would say, has many 
opportunities for improvement in how it interacts with the 
field, from the way it does its studies to making sure that 
people in the field know that it exists. There have been recent 
changes even to its Web site so that when someone goes it is 
organized in terms of frequently asked questions--I want to 
know about how to help my preschooler--and starting to organize 
and revamp the site in that way.
    So it is definitely an area of continuous improvement, but 
the goal of trying to give an objective, clear, easy to 
understand resource is absolutely something that we need.
    Mr. Scott of Virginia. Once you know what works how do you 
get--do you have a training mechanism so teachers can actually 
get trained in what works or is it just sit there, they find 
out what works but don't know how to do it?
    Mr. Kemple. I think the mechanisms for translating the 
evidence into practice are--in many ways are still wanting. I 
think from a policymaking perspective and from the perspective 
of trying to decide how to allocate scarce resources, I think 
one critical lever may very well be ensuring that if a state or 
a district wants to use federal resources or a district wants 
to use state resources, or if a school has access to some 
flexible resources they should be obligated to demonstrate that 
the way that they would like to use those resources has 
appropriate and rigorous evidence behind its effectiveness, and 
that they have a way of being able to link with the people who 
either have produced that evidence or started to translate it 
into guidelines--that they can use to implement their 
programs----
    Mr. Scott of Virginia. Well, but how does--how does a 
teacher get trained in what works? I mean, he or she is sitting 
in the classroom, you read this research works, how do you 
learn how to do it?
    Ms. Long. I think that is where the practice guides come 
in, because that is taking of research and putting it into the 
pieces of the day-to-day job of a teacher, now what do you do 
with this information? How do you actually apply it in your 
classroom? And so that is another translation function that IES 
has taken upon itself.
    Mr. Scott of Virginia. And is it working?
    Ms. Christie?
    Ms. Christie. If I could just make one suggestion that I 
don't hear talked about at all, and that is if every time there 
was a new piece that was put into the What Works Clearinghouse 
or a new conclusion drawn, if that could be pushed out to 
school boards across this country, it is school boards, then, 
that help decide how professional development is delivered, how 
teachers collaborate within the districts, what they should be 
looking at.
    And IES already has the contact information for every 
district. You wouldn't have to have all the boards. The 
districts then could relay that to the----
    Mr. Scott of Virginia. My time is almost expired. I wanted 
to get in another question about whether or not there is 
research on nonverbal communications, which can be very 
important in how children react to education.
    Mr. Kemple. Not something I am very familiar with.
    Mr. Scott of Virginia. You are not?
    Mr. Kemple. Not, no.
    Ms. Christie. I am not sure what you are asking.
    Mr. Scott of Virginia. Nonverbal communications and teacher 
interacting with the student, conveying caring or not caring 
and the children reacting to that. Is there any research on 
that?
    Ms. Christie. I am not familiar with the research on that 
but I am familiar with the tools that IES has to quickly find 
out who else knows or done any research on that.
    Mr. Scott of Virginia. Thank you, Mr. Chairman.
    Mr. Rokita. I thank the gentleman.
    Mr. Holt is recognized for 5 minutes.
    Mr. Holt. Thank you.
    I thank the witnesses.
    This is encouraging. I am hearing a--what I had hoped to 
hear about IES.
    I will begin by saying we badly need rigorous research. We 
have been hampered for, well, decades--maybe forever--because 
every policymaker, every school board member, policymaker on 
the state or federal level was a student and, therefore, is an 
expert on education. And so we end up with the same old, with 
frequent overlays of short-lived fads. And so I hope that we 
can make the most of this rigorous research.
    The proposal came from Mr. Walberg on the other side that 
we devolve this to the states or turn it over to the private 
sector. Let me ask two specific questions: Are most states 
doing rigorous, relevant research? Does anybody----
    Ms. Long. My sense is the answer is no because they don't 
have the capacity to do it.
    Mr. Holt. Okay. Is the private sector doing sufficient 
studies of rigorous, unbiased research that teachers or 
policymakers can trust?
    Mr. Kemple. On their own initiative I would say very few, 
if any.
    Mr. Holt. So the conclusion I draw from that is we need 
this. So let me ask the next question: Do we need more of this? 
Are we close to saturation in producing the kinds of rigorous 
studies that we need to wisely spend many tens of billions of 
dollars every year in actually--well, in education?
    Ms. Long. No. Not at all.
    Monitoring the applications for research proposals over 
time to IES, the quality of them has increased dramatically 
over time. And unfortunately, many of them are not funded, and 
during the last year there were a number of very highly rated 
research proposals through the peer review process that 
unfortunately were not able to be funded.
    So there are many great----
    Mr. Holt. More than were funded? Were a minority of the 
well-reviewed proposals funded?
    Ms. Long. That is correct. That is correct.
    Mr. Holt. Okay. Does the IES review the--have there been 
studies within the IES of the application of former studies? 
Does the IES review the application over the years or the 
follow up that occurs on those studies?
    Ms. Long. There has been a lot of attention. It has looked 
back and it is hard to find out what impact those have had, and 
so this year there is actually a competition to establish a 
center of knowledge utilization just to address that issue as 
well as requiring funded researchers to have dissemination 
plans and communication with the field so that their research 
results are used by those practitioners.
    Mr. Holt. Changing the subject, one of the things we are 
asked to consider reauthorizing would be the state longitudinal 
data systems. Are any of the witnesses expert in the data 
systems? My specific question is, are we learning from the good 
data systems things that are useful in the classroom?
    Mr. Kemple. I think we are just beginning that process. In 
New York City we have formed an explicit research alliance that 
works closely with the New York City public school system and 
have been able to acquire its entire administrative records 
database, along the same lines as what the state longitudinal 
data systems are supposed to be doing across the country.
    It has turned out to be an incredibly valuable tool both 
for research and for policy and practice. I think to the degree 
that we can continue to help states both create these systems 
and then have them link up with the capacity to make use of 
those for research purposes, I think we will quickly see 
mechanisms that will allow that work to penetrate into schools 
and classrooms.
    Mr. Holt. And then I guess this is specifically for Dr. 
Kemple: How can research be embedded in the programs that are 
authorized and funded? How good a job are we doing at it? How 
could we make that happen?
    Mr. Kemple. Again, I think this is something that has a 
long way to go, I think in two ways--one, particularly when 
there are hard decisions about how to make use of resources 
that can be used for innovation. I think any use of resources 
for innovation, be they from the federal, state, or local 
level, should be accompanied either by strong evidence that 
this innovation has the capacity to change teaching and 
learning in schools for the better; or B, if there is lacking 
evidence, that there is a requirement that the participants or 
the recipients of the funding be willing to participate in a 
rigorous research study to establish whether or not those 
resources are paying off in better teaching and learning.
    Mr. Holt. Thank you.
    Mr. Rokita. I thank the gentleman.
    Mr. Hinojosa, you are recognized for 5 minutes.
    Mr. Hinojosa. Thank you, Mr. Chairman. I thank you all for 
inviting this wonderful panel to talk to us about improving 
teaching and learning in classrooms across our country.
    I want to say that I am a strong supporter of the regional 
educational laboratories and the comprehensive center programs. 
My first question is going to go to Dr. Long about the new 
emphasis at the Institute of Education Sciences, you call IES, 
on knowledge utilization, which I think is just a fancy way of 
saying we want to ensure that educators know about the research 
on what works.
    Therefore, can you tell us about the work at IES Sciences 
in this area with early childhood development?
    Ms. Long. With early childhood development specifically? 
Unfortunately, I can't speak specifically to that area. I can 
say, generally the issue has been how do we translate our 
research, research that might be on early childhood education--
while also trying to understand from the practitioners' view, 
what are the questions that they have, what are their pressing 
needs. You have to understand how they do their work so how 
they might actually change practice with the research results, 
so that it can't be just one-directional from IES. What IES 
needs to do and is trying to do is build a two-way street so we 
have communication from the people who work with our young 
children of what they need, what are the pressing questions, 
and how they might--how they do their work and then how the 
research might be able to inform that.
    Mr. Hinojosa. I am going to ask another question and 
probably come back to you, Dr. Long.
    Ms. Christie, you spoke about your learning the importance 
of education from cradle to age 3 years of age, and you used 
your children that you stayed home and educated them at home. 
Why do you think that state and federal elected officials do 
not consider a high priority to invest in early education 
programs?
    And the reason I ask you that question is that years ago 
Buck McKeon was chairman of our Subcommittee on Higher Ed and 
took a codel to China. We were trying to find out why it was 
that they would usually beat the United States on competition--
international scholastic competition. And one of the people who 
answered our question was an older gentleman--a professor--and 
he said it was a very simple answer.
    The formula he said was early reading plus writing equals 
success in school, and that they started reading to the 
children the moment they were born. So they read to them in the 
cradle and then they already knew how to read by the time they 
were 3 years old and pecking on a computer at age 4 to tell us 
what they read. It was that simple.
    So you tell me, was it research that you read that made you 
believe that you wanted to stay home and spend the time you did 
with your children from the time they were born on?
    Ms. Christie. When I was in my preparation program for 
being educated as a teacher, yes, that was part of the program. 
I do believe state policymakers are very compelled to address 
those early years. Sometimes it is simply a fiscal issue, and I 
think we are seeing the investments go up.
    We track state of the state addresses by all the governors 
and I can tell you, it was a prominent part of a number of 
governors this year and that is not unusual. We do see that. So 
they are very--they are not uninvolved in it. They are very, 
very interested and most of the time it is a fiscal issue.
    Mr. Hinojosa. Dr. Long, do you have research that shows 
that it is working, where they invest in the regions of the 
country in the cradle to age 3 and 4 on early childhood 
development?
    Ms. Long. There is certainly a research basis for those 
early investments have long-term impacts. There have even been 
studies that looked at children in preschool and then followed 
them years and years later to see that they were doing better 
in high school and better in college. So there is that research 
basis.
    And I would say in a short amount of time, as we have 
started to collect this information, fund additional 
information, and put it in a central location so we can figure 
out how all the different pieces of the puzzle fit together, we 
are starting to come out with some strong conclusions about 
what does work, although there is so much more that we don't 
understand and so many things we have found that don't work, 
and so we need alternatives.
    Mr. Hinojosa. Thank you, Mr. Chairman. I yield back.
    Mr. Rokita. I thank the gentleman.
    I just have a couple questions.
    Again, I want to thank the witnesses. I appreciate all your 
testimony. Let me start with Mr. Scott.
    Your testimony stated, I believe, that the performance 
measures that IES uses do not reflect current programs, so can 
you provide some specific examples of this?
    Mr. Scott. Mr. Chairman, I do think it is important to note 
that when you look across any number of the current performance 
measures IES has in place, they are actually doing quite well 
in many areas. A couple areas that we have identified where we 
believe there needs to be more transparency and accountability 
includes the RELs as well as some of the new research grant 
programs that IES has not yet developed performance measures 
for those activities.
    Mr. Rokita. And what can we be doing better to--or IES be 
doing better to accurately evaluate its programs? Is that the 
same thing?
    Mr. Scott. I think first of all, involve policymakers and 
practitioners in the research agenda setting; develop relevant, 
timely, up-to-date performance measures; have the information 
necessary to evaluate those activities; and then hold those 
activities accountable.
    Mr. Rokita. What was the second to the last one?
    Mr. Scott. Develop the measures, involve stakeholders in 
the research agenda setting----
    Mr. Rokita. What would a couple examples of performance 
measures be in your mind?
    Mr. Scott. You know, for example, identifying feedback 
opportunities to involve stakeholders, measuring how you are 
doing in relation to what the stakeholders' needs are. They do 
have a number of measures related to the What Works 
Clearinghouse, in terms of the level--the number of 
interventions that have been supported by their activities.
    And so I do think, you know, they have made progress in 
this area, but particularly as it relates to the regional 
educational labs and some of the new grant programs, we do 
believe it is critically important that you establish 
performance measures in those areas--particularly the RELs. 
That is a significant investment on the part of IES, and so to 
not have public reporting and not have public accountability 
around that activity we believe is a key area for improvement.
    Mr. Rokita. And again, what would--in your mind, what would 
that public reporting look like?
    Mr. Scott. As again, you know, having key indicators, 
performance measures, in terms what the expectations are for 
the RELs, but then having feedback----
    Mr. Rokita. What would that be? I am trying to get you to 
be specific and quantitative.
    Mr. Scott. Quantitative. Could be, once again, meeting the 
needs--having a feedback loop in terms of meeting the needs of 
the stakeholders, having clear expectations in terms of how you 
expect the RELs to be engaged with the research alliance, and 
then hold them accountable for that cooperation.
    Mr. Rokita. Okay. I am going to leave you alone.
    [Laughter.]
    Mr. Scott. I don't want to get too prescriptive here. I 
mean, one of the things we----
    Mr. Rokita. Okay. Fair enough.
    Mr. Scott [continuing]. Do believe is having both 
quantitative and qualitative measures in place.
    Mr. Rokita. There we go. You got----
    Mr. Scott. Customer service satisfaction. That is the word 
I was looking for when I was talking about getting feedback. I 
mean, the National Center for Education Statistics does a 
really good job of that, and when we talked to the stakeholders 
that is one of the things they pointed out is that a lot of the 
projects and the products produced by NCES are really useful, 
and there are clearly opportunities for IES to do that across 
more of its activities is to get that direct customer 
satisfaction survey information and use it directly.
    Mr. Rokita. Okay.
    Dr. Long, your testimony--I conclude IES has a broad 
mission to provide useful information to many different 
audiences. In times of limited resources, though, and this was 
touched on a little bit earlier, in your opinion, who can 
benefit most from IES products? Remember, we are broke.
    Ms. Long. Yes. There are many audiences. I think 
policymakers absolutely need the information that is coming 
from IES to make better decisions what to do with limited 
resources, what works, and for what cost.
    I think researchers absolutely sending signals to them 
about how they should spend their time. I don't think IES needs 
to do everything; it needs to leverage the field. And by using 
the signals, the incentives, its convening power to take the 
researchers out in the field, the organizations, whether they 
be states or school boards, to get that information out to 
them.
    I think it also does have a translation responsibility to 
the field to make clear what do you do with this research 
information, but then again, working in concert with other 
organizations and associations to get the information out 
there.
    Mr. Rokita. And for what audience? For policymakers, you 
say, and for who else?
    Ms. Long. I said for policymakers, I said education 
researchers should help direct where they are putting their 
efforts. I said in terms of the field--so teachers, principals, 
superintendents--the translation function of IES is very 
important. But again, to get that out I think it is working in 
concert with organizations and associations that are in the 
field--not for them to necessarily do everything else, but they 
have the unique position of being completely objective, having 
convening power like no one else does, and so they can send 
signals, incentives, bring people together in a very different 
way than any other organization.
    Mr. Rokita. Okay. Fair enough.
    And then in the 30 seconds I have left, if the two 
remaining witnesses would like to respond to either of my 
questions, you are welcome to.
    Dr. Kemple, in about 10 seconds?
    Mr. Kemple. Yes. I think the Congress and the U.S. 
Department of Education I think would be the primary 
beneficiaries of IES's work, both on the quality and the 
quantity of work that gets produced. And in terms of answering 
questions about what--not just about effective practices but 
also about ineffective practices, I think it is the 
responsibility to the public to be able to----
    Mr. Rokita. Thank you.
    Mr. Kemple [continuing]. Invest in learning about whether--
--
    Mr. Rokita. Appreciate that.
    Ms. Christie?
    Ms. Christie. I would like to suggest that folks look at 
the government performance accountability system in Washington, 
go to their site, look at the kinds of metrics they report on 
on their site. It is very impressive, and I wish more folks, 
both state and federal, would do the similar thing.
    Mr. Rokita. Okay. Thank you.
    Again, I thank the witnesses and now I yield to the Ranking 
Member Miller for his closing remarks.
    Mr. Miller. Thank you very much, and I want to thank the 
chairman of the subcommittee, and the Chairman, for holding 
this hearing. I think it is very important.
    You know, I think that the testimony is pretty clear that 
we have developed a much better system than we had before, and 
the question is, how do you hold on to that and how do you 
maintain the integrity of that system and sort of grow the 
confidence of the practitioners, if you will, from governors on 
down all the way to the classroom.
    And, Mr. Scott, how do you get into a situation, as you 
point out on page six here, on this question of the peer review 
process starting to stretch itself out and growing from 117 
days in 2011 to 175 days in 2012? I mean, this can be the death 
of an organization.
    Mr. Scott. And I think that is----
    Mr. Miller. You know, we are in a dynamic system where, at 
our level in our districts people think about this in 9-month 
segments, and what is new has got to show up on a fairly 
regular basis, I mean, and coincide with the needs of the 
users, which are, dictated by the school year if nothing else.
    Mr. Scott. Well, I think that issue sort of points to two 
things that we talk about in our statement, one being the 
importance of sort of balancing the rigorous research with 
ensuring it is timely, and useful, and relevant. And the second 
issue is having IES use--IES gathers a lot of information about 
a lot of things going on under its--or in its purview, but we 
have found that at times there are gaps in how they use that 
information to oversee the research, and the issue of the peer 
review is a perfect example of that.
    It is not clear to us that they were really aware of this 
until we started asking some questions about what was going on 
with the timing here, and now they are paying attention to it. 
Our point to IES would be there are other processes that you 
have in place that you should be constantly monitoring to 
ensure things don't spin out of control or start to grow and 
expand in a way that negatively impacts the end users of the 
information and the research.
    Mr. Miller. So you think this can, in fact, be corrected, 
in terms of getting the peer review condensed--I mean, if 
people have signed on to participate in that process, maybe 
they have signed on to do 1,000 of these, I don't know, but at 
some point you have got to know what your contractors--what the 
capacity is to participate. You may want them for their name, 
for their specialties and their expertise, but if they don't 
allow the time so they can fully participate you have got to 
move on and find, I think, somebody else at some point.
    Mr. Scott. It is important for IES to have the right 
information to monitor the performance and then to take the 
necessary corrective actions to ensure that that peer review 
process doesn't continue to grow and negatively impact its 
ability to provide the research.
    Mr. Miller. Can you just put in,--quickly, your concerns 
about the regional education labs? You come across that in your 
testimony----
    Mr. Scott. If you look historically at some of the 
challenges around the regional education labs, we continue to 
be concerned that there are not clear performance measures for 
the RELs, that it is not publicly available. IES is collecting 
some information. IES also has an ongoing evaluation of the 
previous cohort of the RELs that is due to be completed at some 
point here in the near future.
    We do think it is important, though, for accountability 
reasons that there be some public accounting for those RELs, 
that they have clear performance measures and indicators, and 
then that IES take the necessary action to hold them 
accountable for their performance.
    Mr. Miller. I mean, this is--at least as it has been 
presented--this is supposed to be some high performance 
operation, and the question that you are raising to some extent 
is whether or not that, in fact, you are getting a high 
performance operation in some of the regions----
    Mr. Scott. Well, we have heard comments from certain 
stakeholders that certain RELS are more productive than others, 
the relevance of the research that certain RELs produce is more 
relevant than others. I think the question we have for IES is, 
you know, at what point are we going to have more public 
accounting for the performance of the RELs? That is where you 
start--public accounting for their performance. And then you 
can make the necessary decisions after then how to move 
forward.
    Mr. Miller. Thank you.
    Thank you, Mr. Chairman.
    Mr. Rokita. I thank the ranking member.
    Let me again and finally thank the witnesses for your 
testimony. It was very enlightening. Very much appreciate it.
    I think what we learned is that the quality of education 
research has greatly improved in the last decade, and for that 
we are thankful. And we are thankful in particular for your 
leadership in the field.
    However, like most things in life, there are places that we 
can look to strengthen the evaluation and performance 
requirements for IES programs and ensure that the research 
coming out of it is rigorous, relevant, and useful to education 
practitioners. And I think you saw that in the questioning by 
several members this morning. I think that ought to be our next 
goal and a continuation of the work that we have done.
    So I look forward to working with my colleagues to 
reauthorize the Education Sciences Reform Act in a positive way 
that recognizes the fiscal condition we are in, how to leverage 
not only the resources of IES but its partners at the federal 
and state level, and certainly the leadership of each of the 
witnesses here today.
    And with that, seeing no further business before this 
committee, this hearing is adjourned.
    [Whereupon, at 11:27 a.m., the committee was adjourned.]

                                 

```