Reading First: States Report Improvements in Reading Instruction, but Additional Procedures Would Clarify Education's Role in Ensuring Proper Implementation by States
Published: 2007-02-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Reading First program was designed to help students in kindergarten through third grade develop stronger reading skills. This report examines the implementation of the Reading First program, including (1) changes that have occurred to reading instruction; (2) criteria states have used to award sub-grants to districts, and the difficulties, if any, states faced during implementation; and (3) the guidance, assistance, and oversight the Department of Education (Education) provides states. GAO's study is designed to complement several studies by Education's Inspector General (IG) in order to provide a national perspective on some of the specific issues being studied by the IG. For this report, GAO administered a Web-based survey to 50 states and the District of Columbia, and conducted site visits and interviews with federal, state, and local education officials and providers of reading programs and assessments.
section_title: What GAO Found
paragraphs: States reported that there have been a number of changes to, as well as improvements in, reading instruction since the implementation of Reading First. These included an increased emphasis on the five key components of reading (awareness of individual sounds, phonics, vocabulary development, reading fluency, and reading comprehension), assessments, and professional development with more classroom time being devoted to reading activities. However, according to publishers we interviewed, there have been limited changes to instructional material. Similarly, states report that few changes occurred with regard to their approved reading lists. States awarded Reading First sub-grants using a variety of different eligibility and award criteria, and some states reported difficulties with implementing key aspects of the program. After applying federal and state eligibility and award criteria, Education reported that over 3,400 districts were eligible to apply for sub-grants in the states' first school year of funding. Of these districts, nearly 2,100 applied for and nearly 1,200 districts received Reading First funding. Education officials made a variety of resources available to states during the application and implementation processes, and states were generally satisfied with the guidance and assistance they received. However, Education developed no written policies and procedures to guide Education officials and contractors in their interactions with state officials and guard against officials mandating or directing states' decisions about reading programs or assessments, which is prohibited by the No Child Left Behind Act (NCLBA) and other laws. Based on survey results, some state officials reported receiving suggestions from Education officials or contractors to adopt or eliminate certain reading programs or assessments. Similarly, the IG reported in September 2006 that the Department intervened to influence a state's and several school districts' selection of reading programs. In addition, while Education officials laid out an ambitious plan for annual monitoring of every state's implementation, they did not develop written procedures guiding monitoring visits and, as a result, states did not always understand monitoring procedures, timelines, and expectations for taking corrective actions.

section_title: Letter
paragraphs: The Reading First program, signed into law in 2002 as part of the No Child  The Reading First program, signed into law in 2002 as part of the No Child  Left Behind Act (NCLBA), was designed based on reading research to help  Left Behind Act (NCLBA), was designed based on reading research to help  children in kindergarten through third grade develop stronger reading  children in kindergarten through third grade develop stronger reading  skills. Since 2002, Congress has appropriated about $1 billion a year for  skills. Since 2002, Congress has appropriated about $1 billion a year for  Reading First, more than for any other federal reading program, to fund 6- Reading First, more than for any other federal reading program, to fund 6- year grants to states. States, in turn, have awarded sub-grants to school  year grants to states. States, in turn, have awarded sub-grants to school  districts to establish reading programs. These sub-grants are targeted to  districts to establish reading programs. These sub-grants are targeted to  districts and schools with the highest percentage or numbers of students  districts and schools with the highest percentage or numbers of students  in kindergarten through third (K-3rd) grade reading below grade level and  in kindergarten through third (K-3rd) grade reading below grade level and  to districts and schools with large numbers of low-income students.  to districts and schools with large numbers of low-income students.  However, in awarding sub-grants, states do have some discretion to  However, in awarding sub-grants, states do have some discretion to  establish priorities, such as giving priority to districts that demonstrated  establish priorities, such as giving priority to districts that demonstrated  leadership and commitment to improving reading.  leadership and commitment to improving reading.
The Reading First program places a number of requirements on state and  The Reading First program places a number of requirements on state and  local grant recipients to ensure they adopt reading programs and methods  local grant recipients to ensure they adopt reading programs and methods  that are effective, while, at the same time, NCLBA provisions restrict the  that are effective, while, at the same time, NCLBA provisions restrict the  ability of Department of Education (Education) officials to mandate or  ability of Department of Education (Education) officials to mandate or  direct state and local decisions on reading curriculum. In particular,  direct state and local decisions on reading curriculum. In particular,  Reading First requires states and participating school districts to adopt  Reading First requires states and participating school districts to adopt  scientifically-based reading programs containing key instructional  scientifically-based reading programs containing key instructional  components identified in the law. In some instances, state officials have  components identified in the law. In some instances, state officials have  required prospective Reading First districts to choose their reading  program from a state-approved list. In contrast, other states have opted to  vest district applicants with responsibility for researching and selecting  the reading programs to be used in their Reading First schools. States and  participating school districts are also required to provide professional  development in reading that is based on scientific research and to track  students’ progress in reading using valid and reliable assessments.  However, NCLBA and various other statutory provisions place limits on  what Education officials can require states, districts, and schools to do.  For example, under NCLBA, Education officials are not authorized to  mandate, direct, or control state, school district or school curriculum or  program of instruction. In other words, these provisions are intended to  preserve state and local control over key aspects of the public school  system.
During the implementation of the Reading First program some concerns  were raised by publishers of reading programs to Congress and others  about Education’s interactions with state and local grantees regarding  reading programs and assessments. Specifically, several groups  representing reading programs filed complaints with Education’s  Inspector General alleging that Education officials, contractors, and  consultants pressured state and local applicants to choose specific reading  programs and assessments—actions that are expressly prohibited by  NCLBA.
In response to these allegations, Education’s Inspector General (IG)  undertook several separate investigations and audits of various aspects of  the program’s administration.  For the first of these reports, issued in  September 2006, the IG reviewed various aspects of the application  process for awarding Reading First grants to states, including the selection  and composition of the expert panel that reviewed state applications,  application review and feedback procedures, and interactions between  Education and state-level officials.  During its review, the IG, among other  things, interviewed federal and selected state officials and reviewed  departmental guidance, internal e-mail correspondence, and state  applications from 11 states and 1 territory.  Among its findings, the IG  reported that the Department intervened to influence a state’s and several  school districts’ selection of reading programs, actions that call into  question whether program officials violated statutory prohibitions against  directing or controlling state and local curricular decisions.  The IG made  a number of related recommendations, including that the Department  should (1) develop internal management policies and procedures to ensure  that programs are managed in compliance with applicable laws and  regulations, (2) review the management and staff structure of the Reading  First program office, and (3) develop guidance for program officials on  statutory prohibitions on intervening in state and local curricular  decisions.  The Secretary of Education concurred with all of the report’s  recommendations.  In their written comments to the IG report, Education  officials detailed numerous action steps they planned to undertake in  response to the report’s findings and recommendations, including the  appointment of new internal leadership for the Reading First program,   the development of annual training on internal controls, and the drafting  of a memorandum for all program managers regarding the importance of  impartial job performance.
GAO’s study is designed to complement the IG’s work by providing a  national perspective on some of the specific issues being studied by the  IG, as well as providing some information on how the program has been  implemented at the state level, primarily. To obtain a better understanding  of how the Reading First program is being implemented at the state level  and to determine how Education conducts its interactions with states and  the kind of guidance and oversight it provides, GAO agreed to answer the  following questions:  1.  What changes have occurred to reading instruction since the inception  of Reading First?  2.  What criteria have states used to award Reading First sub-grants to  districts, and what, if any, difficulty did states face in implementing the  program?  3.  What guidance, assistance, and oversight did Education provide states  related to the Reading First program?
To answer these questions, we collected information about the Reading  First program from a variety of sources. We conducted a Web-based  survey of the Reading First Directors in all 50 states and the District of  Columbia, and 100 percent responded. We also obtained and analyzed data  from the Department of Education for each state on Reading First  districts’ eligibility, applications, and awards for states’ first school year of  funding. To assess the reliability of this data, we talked to agency officials  about data quality control procedures and reviewed relevant  documentation. We also conducted semi-structured follow-up interviews  with Reading First Directors in 12 states and conducted site visits to 4 of  the 12 states. During the site visits, we met with state and district officials,  as well as school officials, including teachers, principals, and Reading First  coaches. We selected these 12 states to represent diversity in terms of  geographic distribution, grant size, poverty rates, and student reading  achievement, as well as the presence or absence of a statewide list of  approved reading programs. In our selection of the 12 states, we also took  into consideration whether or not state officials reported that they had  received guidance from Education officials advocating for or against  particular reading programs or assessments. For both the survey and  follow-up interviews, we agreed to protect the respondents’ confidentiality  to encourage candid responses: therefore; we will not identify states by  name in this report. We also interviewed administrators from each of the  three regional Reading First Technical Assistance Centers (TAC), officials  from RMC Research—which provided technical assistance to states during  the application process and oversaw the TACs, and the American  Institutes for Research (AIR)—the contractor responsible for conducting  monitoring visits in each state. In addition, we interviewed several  publishers and other providers of reading curricula and assessments, as  well as Reading First officials and attorneys at Education. We also  reviewed state grant files, monitoring reports, and related correspondence  for the 12 states where we conducted follow-up interviews, as well as  legislation and guidance related to Reading First. We also interviewed  Education officials about the implementation of the Reading First  program. Finally, we reviewed Education’s study “Reading First  Implementation Evaluation: Interim Report” to identify information  collected at the district and school level that related to the findings from  our state survey and also the two relevant reports that had been issued by  Education’s Inspector General at the time our audit work was completed.
See appendix I for additional information on the survey, as well as our  other data collection methods. We conducted our work from December  2005 through January 2007 in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: Reading First, which was enacted as part of NCLBA, aims to assist states  and local school districts in establishing reading programs for students in  kindergarten through third grade by providing funding through 6-year  formula grants. The goal of the program is to ensure that every student can  read at grade level or above by the end of third grade. To that end, Reading  First provides funds and technical assistance to states and school districts  to implement programs supported by scientifically-based reading research  (SBRR), increase teacher professional development based on this  research, and select and administer reading assessments to screen,  diagnose, and monitor the progress of all students. NCLBA defines SBRR  as research that (1) uses systematic, empirical methods that draw on  observation or experiment; (2) involves rigorous data analyses that test  stated hypotheses and justify general conclusions; (3) relies on  measurements or observational methods that are valid; and (4) has been  accepted by a peer-reviewed journal or approved by a panel of  independent experts. Further, NCLBA requires states to adopt reading  programs that contain the five essential components of reading-- (1)  phonemic awareness; (2) phonics; (3) vocabulary development;   (4) reading fluency, including oral reading skills; and (5) reading  comprehension strategies.
While Education has responsibility for overseeing the Reading First  program and states’ implementation and compliance with statutory and  program requirements, NCLBA places restrictions on what Education  officials can require states to do. Specifically, Education is not authorized  to mandate, direct, control, or endorse any curriculum designed to be used  in elementary or secondary schools. Further, when Education was formed  in 1979, Congress was concerned about protecting state and local  responsibility for education and, therefore, placed limits in Education’s  authorizing statute on the ability of Education officials to exercise any  direction, supervision, or control over the curriculum or program of  instruction, the selection of textbooks or personnel, of any school or  school system.
Every state could apply for Reading First funds, and states were required  to submit a state plan for approval that demonstrates how they will ensure  that statutory requirements will be met by districts. Education, working in  consultation with the National Institute for Literacy (NIFL), as required in  NCLBA, established an expert review panel composed of a variety of  reading experts to evaluate state plans and recommend which plans  should be approved. In these plans, states were required to describe how  they would assist districts in selecting reading curricula supported by  SBRR, valid and reliable reading assessments, and professional  development programs for K-3rd grade teachers based on SBRR.  The law  does not call for Education to approve or disapprove particular reading  programs or curricula identified in state plans.  When appropriate, the peer  review panel was to also recommend clarifications or identify changes it  deemed necessary to improve the likelihood of a state plan’s success.  NCLBA requires that Education approve each state’s application only if it  meets the requirements set forth in the law.
Reading First allows states to reserve up to 20 percent of their funds for  professional development; technical assistance; and planning,  administrative, and reporting activities. For example, states can use their  funds to develop and implement a professional development program to  prepare K-3rd teachers in all essential components of reading instruction.  One model for supporting teachers’ reading instruction involves hiring a  Reading Coach who works with teachers to implement reading activities  aligned with SBRR. Almost all states require Reading First schools to have  a Reading Coach tasked with supporting teachers and principals with  instruction, administering assessments, and interpreting assessment data.
States that receive Reading First grants are required to conduct a  competitive sub-grant process for eligible school districts and must  distribute at least 80 percent of the federal Reading First grants they  receive to districts. NCLBA and Education guidance provides states with  flexibility to set eligibility criteria for school districts so that eligible  districts are among those in the state that have the highest number or  percentage of K-3rd grade students reading below grade level and (1) have  jurisdiction over an empowerment zone or enterprise community, (2)  have a significant number or percentage of schools identified as in need of  improvement, or (3) are among the districts in the state that have the  highest number or percentages of children counted as poor and school- aged for the purposes of Title I. NCLBA establishes priorities that states  must consider when awarding a Reading First sub-grant, while also  allowing states to establish other priority areas. For instance, NCLBA  requires that the state sub-grant process give priority to districts with at  least 15 percent of students or 6,500 children from families with incomes  below the poverty line, but states also have some flexibility to establish  additional priorities, such as a demonstrated commitment to improving  reading achievement. The sub-grant process along with the criteria at each  stage is summarized in figure 1.
Districts are required to use their sub-grant funds to carry out certain  activities identified in NCLBA. For example, districts must use these funds  to select and implement reading programs based on SBRR that include the  essential components of reading instruction, to select and implement  diagnostic reading assessment tools, and to provide professional  development opportunities for teachers. Additionally, districts are  permitted to use Reading First funds in support of other activities, such as  training parents and tutors in the essential components of reading  instruction.
States are required to report to Education annually on the implementation  of Reading First, including their progress in reducing the number of  students who are reading below grade level. Additionally, states are  required to submit a mid-point progress report to Education at the end of  the third year of the grant period. These mid-point progress reports are  subject to review by the same expert peer review panel that evaluated  state applications. If Education determines, after submission and panel  review of a state’s mid-point progress report and on the basis of ongoing  Education monitoring, that a state is not making significant progress,  Education has the discretion to withhold further Reading First grant  payments from that state. While these state reports to Education are  intended to provide information on the effectiveness of Reading First,  Education is also required to contract with an independent organization  outside Education for a rigorous and scientifically-valid, 5-year, national  evaluation of the program, with a final report scheduled to be issued in  2007.
The Reading First program has relied on several key contractors to  perform a number of program functions. For example, Education officials  hired RMC Research Corporation, a company that provides research,  evaluation, and related services to educational and human services clients,  to provide technical assistance to states and districts that have received  Reading First funding. According to Education officials, RMC contractors  were tasked initially with providing specific, individualized guidance on  the application process to state officials who requested it. RMC later  became the national coordinator for the contract overseeing the National  Center for Reading First Technical Assistance and its three regional  subsidiaries: the Eastern Regional Reading First Technical Assistance  Center (ERRFTAC) in Tallahassee, Florida; the Central Regional Reading  First Technical Assistance Center (CRRFTAC) in Austin, Texas; and the  Western Regional Reading First Technical Assistance Center (WRRFTAC)  in Eugene, Oregon. In this role, RMC staff provides support to the TACs  and their employees, as well as weekly coordination among the TACs, and  regular training seminars. Operated out of universities recognized by  Education officials for their expertise in SBRR and related areas, the  centers began operations in 2003 and are responsible for providing an  array of technical assistance activities to states, including national and  regional conferences, training and professional development, products and  materials, and liaisons to national reading experts. Education officials also  contracted with Learning Point Associates to provide technical assistance  to states as they launched their sub-grant competitions.
Once Reading First sub-grants had been awarded to local districts,  Education contracted with the American Institutes for Research (AIR), a  behavioral and social science research organization, to conduct annual  monitoring visits to each state. These visits incorporate sessions with state  officials, as well as visits to a few districts in each state and are designed  to assess states’ and districts’ compliance with their approved plans. After  each monitoring visit, AIR representatives submit a report, including any  findings of non-compliance, to Reading First officials. Reading First  officials are to forward these reports to the cognizant state officials.
section_title: States Reported Changes as Well as Improvements in Reading Instruction Since the Inception of Reading First
paragraphs: States reported that there have been a number of changes and  improvements in reading instruction since the implementation of Reading  First. There has been an increased emphasis on the five key components  of reading, assessments, and professional development with more  classroom time being devoted to reading activities. However, according to  publishers we interviewed, there have been limited changes to  instructional material. Similarly, many states that approved reading  programs for districts to choose from report few changes to their lists of  approved programs.
subsections: section_title: States Reported That Reading Instruction Changed or Improved Following the Implementation of Changes under Reading First
paragraphs: In responding to our survey, 69 percent of all states reported great or very  great improvement in reading instruction since inception of Reading First.  One area in which states reported a change that may have contributed to  improvement of reading was the degree to which classroom instruction  explicitly incorporated the five key components. In our survey, at least 39  states reported that Reading First schools had incorporated each of the  five required components of reading into curriculum to a great or very  great degree as a result of Reading First. State and local officials we talked  to during some of our site visits reinforced this opinion and in particular  noted that Reading First teachers had awareness of and were more  focused on the five components. In addition, the increased time devoted to  reading activities under Reading First may have contributed to  improvement. Several district officials we met with told us they were  including a protected, uninterrupted block of time for reading instruction  of 90 minutes or more per day—which the department’s Guidance for the  Reading First Program lists as a key element of an effective reading  program. Education’s Reading First Implementation Evaluation: Interim  Report (The Interim Report) also found that Reading First teachers  reported allocating over 90 minutes per day, on average, for a designated  reading block.
States officials reported improvement in reading instruction resulting from  the use of assessments. In responding to our survey, one state official  said, “One of the strengths of the Reading First program has been its  strong adherence to SBRR and to the use of valid and reliable assessments  in guiding instruction and program evaluation.” A number of state and  local officials we interviewed reported that the use of assessments  changed after Reading First, especially in the way that teachers use data  from these assessments to better inform reading instruction. Specifically,  district officials we talked to during our site visits reported that teachers  review students’ assessment results to determine the areas in which they  need more targeted instruction. One official also reported that assessment  data can sometimes be used to identify successful teachers from whom  other teachers can learn teaching techniques, with one official asserting  that “Reading First has and is making a great impact on teachers’  instructional practices, techniques, and strategies.” Also, according to  Education’s Interim Report, researchers estimated that 83 percent of  Reading First teachers cited assessment results essential to organizing  instructional groups, 85 percent cited the results essential to determining  progress on skills, and 75 percent cited the results essential to identifying  students who need reading intervention.
According to our survey, most states also reported that the assessments  they used differed greatly or very greatly from the ones they used prior to  Reading First. States reported a wide variety of reading assessments on  their state-approved lists, with over 40 different assessments listed. By far,  the most frequently approved assessment was Dynamic Indicators of Basic  Early Literacy Skills (DIBELS), approved by 45 states. Also, a few states  reported to us that they were moving toward a more uniform systematic  assessment system for the first time, whereas previously each school  could choose which assessment it would use. Some state and district  officials told us that having a more uniform and systematic assessment  was beneficial, because, for instance, it allowed the officials to track and  compare reading scores more easily.
Professional development is another area in which state officials noted  improvement. All states reported improvement in professional  development as a result of Reading First, with at least 41 states reporting  that professional development for reading teachers improved greatly or  very greatly in each of five key instructional areas. Further, a considerable  majority of states reported great or very great increases in the frequency of  professional development and the resources devoted to it, 45 and 39,  respectively. One state reported, “The provision of funding to be used to  support statewide professional development efforts for K-3 reading has  been an important aspect of the program.” The Interim Report on the  Reading First program highlights that a vast majority of Reading First  teachers had received training on the five key components of reading. In  our site-visits, district officials confirmed that, for the most part, teachers  in their Reading First classrooms had received training. However, in  responding to our survey, 19 states did report some challenges in training  of 100 percent of Reading First teachers, with teacher turnover cited by 12  states as the reason some Reading First teachers might not have taken any  type of Reading First training. Figure 2 summarizes reported  improvements in professional development for teachers.
Professional development was provided by a variety of federal, state, and  private sources. Staff from the TACs and officials from at least one state  reported providing professional development to districts customized to  the individual district’s needs and perceived future needs. Education’s  Interim Report on Reading First implementation noted that state Reading  First coordinators in 33 states reported that state staff chose and  organized all statewide professional development efforts and played a key  role in selecting professional development topics for districts and schools.  In addition, publishers we spoke with told us they often provide training to  acclimate teachers to their products. Certain publishers of major  commercial reading programs and assessments told us that since the  implementation of Reading First, districts demand much more training.  Specifically, according to some of the publishers and TAC staff we spoke  with, districts have been interested in more in-depth workshops on  particular topics such as teaching techniques and using and interpreting  assessments.
Finally, another aspect of professional development pertinent to Reading  First is the presence of a Reading Coach. State and district officials  reported that Reading Coaches receive training that better enables them to  assist schools. Education’s Interim Report found that each Reading Coach  worked with an average of 1.2 schools and with 21 teachers to help  implement activities aligned with SBRR.
section_title: Publishers Reported Making Few Changes to Reading Programs, and States Reported Adopting a Variety of Approaches to Select Reading Programs
paragraphs: Three of the four major publishers of reading programs we spoke with  reported that they had not made significant changes to the content of their  reading programs as a result of Reading First. Two publishers stated that  they made minor changes to their reading materials to make more explicit  how the content of the existing programs align with the five components  emphasized in Reading First. Two of them reported that they made  changes to their programs based on the National Reading Panel’s findings,  which was prior to the enactment of Reading First. For example,  representatives of one company stated that they launched a new reading  program based on the findings of the National Reading Panel that takes  into account the requirements of Reading First. Despite limited changes to  the actual instructional material, all the publishers noted a greater  emphasis on assessing the efficacy of their reading programs as a result of  Reading First. In an effort to measure the effectiveness of their programs,  the publishers reported devoting more effort to research and to evaluate  how effective their reading programs were at raising reading assessment  scores.
States followed two main approaches in selecting reading programs for  districts —22 identified a state-approved list of programs for districts to  select, while the other 29 did not have a state-approved list, thereby  requiring districts in those states to self-select reading programs and  determine, with some state oversight and subject to state approval,  whether they satisfy the requirements of SBRR. Of the 22 states with  approved lists, reading program publishers most frequently represented on  the lists were Houghton Mifflin, McGraw-Hill, and Harcourt (see table 1).  At the school level, Education found in its Interim Report that these three  reading program publishers were also the most frequently used, estimating  that between 11 and 23 percent of schools used programs from one of  them.
Additionally, of the 22 states that identified a list of approved core reading  programs for Reading First, 8 already had a list of approved core reading  programs for adoption by all schools in their state prior to Reading First.  Only two of these states reported removing reading programs—a total of  six—from their lists because they did not meet Reading First  requirements. According to Education’s Interim Report, an estimated 39  percent of Reading First schools reported adopting a new core reading  program at the beginning of the 2004-2005 school year in which they  received their Reading First grant, in contrast with an estimated 16 percent  of non-Reading First Title I schools.
States used a variety of sources to help them identify and select reading  programs that met Reading First’s criteria. For example, 15 of the 22 states  with state-approved lists reported using the Consumer’s Guide to  Evaluating A Core Reading Program Grades K-3: A Critical Elements  Analysis to make this decision. Other frequently used resources include  criteria in the state’s application for Reading First, information obtained at  Reading First Leadership Academies provided by Education, and other  states’ approved lists. Based on responses to our survey, the table below  summarizes approaches states used to develop their approved lists (see  table 2).
Based on our survey results, 25 of the 29 states reporting that they did not  have a list of approved core reading programs said they provided guidance  for districts and schools to identify core reading programs. Fifteen of these  states reported directing districts and schools to conduct a review of  reading programs using A Consumer’s Guide to Evaluating a Core Reading  Program. Other states reported providing a variety of guidance to districts  to help them select reading programs supported by SBRR, including  referring them to the approved lists of other states and reviews conducted  by academic experts.
section_title: States Awarded Reading First Sub- Grants to School Districts Using a Variety of Different Criteria, and Some States Reported Difficulties with Implementation
paragraphs: States varied in how they exercised their flexibility to set additional  eligibility and award criteria as allowed by the Reading First program, and  some states reported difficulty with implementing key aspects of the  Reading First program while other states did not. In the areas in which  they were given flexibility, states used a variety of criteria for determining  eligibility and in awarding sub-grants to eligible districts, such as awarding  grants to districts that had previously received federal reading dollars.  Education reported that over 3,400 school districts were eligible to apply  for Reading First sub-grants in the states’ first school year of funding.   Of these districts, nearly 2,100 applied for and nearly 1,200 received  Reading First sub-grants in the states’ first school year of funding.  In  addition, 22 states reported that it was difficult or very difficult to help  districts with reading scores that had not improved sufficiently. On the  other hand, 28 states reported that it was easy or very easy to determine  whether districts’ applications met criteria for awarding sub-grants.
subsections: section_title: States Varied in How they Exercised Their Flexibility to Set Additional School District Sub-grant Eligibility and Award Criteria
paragraphs: States varied in how they exercised their flexibility to set school district  eligibility criteria for sub-grants. The Reading First program provides  states with some flexibility to define eligibility criteria within the statutory  guidelines. For instance, while Reading First requires that states target  districts with students in kindergarten through third grade reading below  grade level, states have flexibility to set eligibility criteria based on the  percentage and/or number of these students within districts. While 34  states reported electing to base eligibility on a percentage of schools with  students reading below grade level, 18 states reported electing to base  eligibility on a number of students reading below grade level. After  applying eligibility criteria, Education reported that states determined that  over 3,400 school districts were eligible to apply for Reading First sub- grants for states’ first school year of funding, or about 20 percent of all  school districts nationwide. However, the percentage of eligible districts  varied greatly across the states, ranging from about 3 to 93 percent.
Of those districts eligible to apply, 62 percent, or nearly 2,100 districts, did  so, as summarized in figure 3 below. States reported a variety of reasons  why eligible school districts did not apply such as the prescriptive nature  of the program, differences in educational philosophy, and inadequate  resources for the application process. For example, officials from a few  states reported that some districts did not have the capacity to write the  grant application. An official from one state reported that some districts  did not have the time and the staff to complete the sub-grant process.  Furthermore, an official from another state reported that the application  process was too lengthy and time-consuming to complete.
Nineteen states reported in our survey that they exercised flexibility in  establishing priorities when awarding Reading First sub-grants. States set  a variety of additional priorities for awarding grants to school districts.  For instance, six states reported that they gave priority to districts that  already had other grants, such as Early Reading First grants, or indicated  that they could somehow use their Reading First funds in combination  with other resources to maximize the number of students reading at grade  level. In contrast, two states gave priority to districts that had not received  other grant funding. In addition, two states gave priority to districts based  on the population of Native Americans or students with limited English  proficiency. After applying selection criteria, states awarded Reading First  sub-grants to about 34 percent or nearly 1,200 school districts for states’  first school year of funding. This represented about 56 percent of the 2,100  eligible districts that applied and nearly 7 percent of all school districts  nationwide for states’ first school year of funding (see fig. 3).
section_title: Some States Reported Difficulty Implementing Key Aspects of the Program
paragraphs: Some states reported difficulty in implementing key aspects of the Reading  First program. Twenty-two states reported that it was either difficult or  very difficult to help districts with reading scores that had not improved  sufficiently. Officials from one state said that this was difficult because it  requires close examination of students reading deficiencies and the  commitment of school leadership. Officials from another state reported  some difficulty in improving selected reading skills of students with  limited English proficiency, which are concentrated in pockets around the  state. Seventeen states reported that it was either difficult or very difficult  to assess how districts applied SBRR in choosing their reading program.  Finally, seven states reported difficulty implementing four or more of six  key program aspects listed in our survey and shown in figure 4. Officials  from one of these states told us that the difficulty with implementation  was due to the newness of the program for which everything had to be  developed from scratch.
On the other hand, states reported ease implementing other key aspects.  In particular, 28 states reported that it was easy or very easy to determine  whether districts’ applications met criteria for awarding sub-grants. For  example, states are required to determine whether districts will adhere to  the key components of the program, such as developing a professional  development program or using reading assessments to gauge performance.  Several states we interviewed suggested that it was easy to make this  determination because some of the Reading First requirements were  already in place in their states before Reading First was implemented. For  example, some state officials we interviewed mentioned using reading  assessments prior to Reading First. In addition, officials in one state told  us that they already had a professional development program in place to  train teachers on the state’s reading program. Twenty-four states reported  that it was easy or very easy to identify reading programs based on SBRR.
section_title: Education Provided a Wide Range of Guidance, Assistance, and Oversight, Generally Satisfying State Officials, but Education Lacked Controls to Guard against Mandating or Endorsing Curricula and Did Not Provide Written Monitoring Procedures
paragraphs: Education officials provided states a wide variety of guidance, assistance,  and oversight, but Education lacked written procedures to guide its  interactions with the states and provided limited information on its  monitoring procedures. Education’s guidance and assistance included  written guidance, preparatory workshops, feedback during the application  process, and feedback from monitoring visits. Additionally, guidance and  assistance were provided by Education’s contractors, including the  regional technical assistance centers. For the most part, state officials  characterized the guidance and assistance they received from Education  officials and contractors, especially the regional technical assistance  centers, as being helpful or very helpful, and many also reported relying on  the expertise of Reading First officials in other states. However, Education  lacked controls to ensure that its officials did not endorse or otherwise  mandate or direct states to adopt particular reading curricula. For  example, according to state officials, Education officials and contractors  made suggestions to some states to adopt or eliminate certain reading  programs, assessments, or professional development providers. In  addition, some state officials reported a lack of clarity about key aspects  of the annual monitoring process, including time frames and expectations  of states in responding to monitoring findings.
subsections: section_title: Education Provided a Wide Range of Guidance and Assistance
paragraphs: Education provided a variety of written and informal guidance and  assistance to states to help them prepare their applications. For example,  three months after the enactment of NCLBA in January 2002, Education  issued two key pieces of written guidance to states pertaining to the  Reading First program and grant application process: the Guidance for the  Reading First Program and Criteria for Review of State Applications.  Education officials also sponsored three Reading Leadership Academies in  the early part of 2002. The Academies were forums for state education  officials to obtain information and build their capacity to implement key  aspects of the Reading First program, including professional development  and the application of SBRR. Education contracted with RMC Research  Corporation to provide technical assistance to states related to the grant  application process. States reported seeking guidance from RMC on  various aspects of the Reading First application, in particular the use of  instructional assessments (17 states) and instructional strategies and  programs (14 states). Throughout the application process, both Education  and RMC officials were available to address states’ questions. In particular,  Education officials provided feedback to states on the results of expert  review panel evaluations of their applications. Consequently, a large  number of states reported that Education required them to address issues  in their applications, most commonly related to the use of instructional  assessments (33 states) and instructional strategies and programs (25  states).  See figure 5 for issues raised about state applications. Forty-eight  states reported that they needed to modify their application at least once,  and 27 reported modifying them three or more times.
Once grants were awarded, Education continued to provide assistance and  contracted with RMC Research to oversee three regional TACs to help  states implement Reading First. RMC established three TACs affiliated  with state university centers in Florida, Texas, and Oregon, which RMC  and TAC officials told us were selected based on their expertise in one or  more areas central to the success of the Reading First program, such as  professional development or reading assessment. Each technical  assistance center was responsible for providing comprehensive support to  each of the states in its geographic region (see fig. 6). States reported that  they looked to these centers for guidance on a variety of issues, especially  creating professional development criteria, using reading assessments, and  helping districts with reading scores that had not improved sufficiently.
According to TAC staff, some of the most common requests they receive  pertain to the use and interpretation of assessment data and use of  Reading Coaches. TAC staff also told us that they catalog recurring issues  or problems.
In addition, according to one RMC official and some state officials, the  TACs provided support to states during implementation to help them  supplement their capacity and expertise in evaluating whether or not  reading programs proposed by districts were based on SBRR. For  instance, staff from the TAC in Florida explained that some states in their  region had asked for assistance in evaluating reading programs that had  been in use prior to Reading First to gauge their compliance with the  requirements of Reading First. Staff from the TAC emphasized that in  reviewing these reading programs, they used the criteria in each state’s  approved state plan as the criteria for determining compliance with  Reading First requirements. Officials in one state explained that while the  staff at their state educational agency (SEA) possessed the knowledge  necessary to conduct reviews of reading programs, scarce state staff  resources would have made it difficult to conclude the reviews in the short  time frame available. Though Education officials were aware of and  initially condoned the TAC review process, Education officials advised all  TACs to discontinue reviews of programs—to avoid the appearance of  impropriety—after allegations were raised about Reading First officials  expressing preference for specific reading programs. (Table 3 provides a  summary of the types of guidance and assistance provided by Education  and its contractors.)
During the application and implementation phases of the Reading First  program, many states came to rely on other unofficial sources of guidance,  including other states’ Reading First officials, in addition to the written  guidance provided by Education. For example, as noted earlier, among the  22 states that had an approved list of reading programs for Reading First  districts, 15 reported using A Consumer’s Guide to Evaluating a Core  Reading Program to assist them in reviewing potential reading programs.  In addition, officials from 21 states reported that other states’ Reading  First Coordinators provided great or very great help during the Reading  First state grant application process. Further, a number of state officials  reported using the information from other states’ websites, such as  approved reading programs, to help inform their own decisions pertaining  to the selection of reading programs. One state official explained, “With  our limited infrastructure and dollars, we were never able to muster the  resources needed to run an in-house programs review,” and further that,  “It worked well for us to use the programs and materials review results  from larger states that ran rigorous review processes.” Another state  official reported that the state did not feel equipped to apply the principles  of SBRR in evaluating reading programs and responded by comparing one  state’s review and subsequent list of reading programs to those of a few  other states to make judgments about allowable programs.
section_title: States Were Generally Satisfied with the Guidance and Assistance Available to Them
paragraphs: Most states reported making use of and being satisfied with the primary  sources of guidance available to them over the course of the Reading First  application and implementation processes. For example, 46 states  reported making use of the two key pieces of Education’s written guidance  in preparing their Reading First applications. A majority of states also  reported that these pieces of guidance provided them with the information  needed to adequately address each of the key application components. For  example, over 40 states reported that the guidance related to the definition  of sub-grant eligibility and selection criteria for awarding sub-grants  helped them adequately address these areas in their application. However,  officials in eight states reported that the guidance on the use of  instructional assessments did not provide them with the information  needed to adequately address this area. (See fig. 7.)
Overall, most state officials were also satisfied with the level of assistance  they received from Education staff and their contractors in addressing  issues related to the Reading First application and implementation  processes. For example, state officials in 39 states reported that Education  staff were of great or very great help during the application or  implementation process. Additionally, officials from 48 states reported  that Education officials were helpful or very helpful in addressing states’  implementation-related questions, which frequently dealt with using  reading assessments and helping districts with reading scores that had not  improved sufficiently. A number of state officials reported to us that they  appreciated the guidance and attention they received from Reading First  officials at Education. For example, one state Reading First Coordinator  reported, “the U.S. Department of Education personnel have been  wonderful through the process of implementing Reading First. I can’t say  enough about how accessible and supportive their assistance has been.”  Another state official remarked that the state’s efforts to make reading  improvements “would have been impossible without their [Education  officials and contractors] guidance and support.” Even officials from one  state who had a disagreement with Education over its suggestion to  eliminate a certain reading program characterized most of the guidance  they received from Reading First officials as “excellent.” However, one  state official reported feeling that the technical assistance workshops have  served as conduits for Education officials to send messages about the  specific reading programs and assessments they prefer. Another state  official reported that, “core programs and significant progress have not  been defined” and that “SBRR programs are not clearly designated.”
According to responses obtained to our survey, the three TACs also  provided a resource for states seeking advice on issues pertaining to the  implementation of their Reading First programs. Specifically, 41 states  cited the Centers as helpful or very helpful in addressing states’ inquiries  related to the implementation of Reading First. In addition, on a variety of  key implementation components, more state officials reported seeking  information from their regional TACs than they did from Education  officials (see table 4).
section_title: Education Lacked Written Procedures to Guide Its Interactions with States
paragraphs: We found that Education developed no written guidance, policies, or  procedures to direct or train Education officials or contractors regarding  their interactions with the states. Federal agencies are required under the  Federal Managers’ Financial Integrity Act of 1982 to establish and  maintain internal controls to provide a reasonable assurance that agencies  achieve objectives of effective and efficient operations, reliable financial  reporting, and compliance with applicable laws and regulations. When  executed effectively, internal controls work to ensure compliance with  applicable laws and regulations by putting in place an effective set of  policies, procedures, and related training. We found that Education had  not developed written guidance or training to guide managers on how to  implement and comply with statutory provisions prohibiting Education  officials from directing or endorsing state and local curricular decisions.  Department officials told us that it was their practice that program  managers should consult the Office of General Counsel if they had  questions regarding interactions with grantees. Reading First officials told  us that it was their approach to approve each state’s method and rationale  for reviewing or selecting reading programs as outlined in each state’s plan  and that state compliance with program requirements, including  adherence to the principles of SBRR, would then be assessed using the  provisions of these plans as the criteria. Similarly, officials from  Education’s contractors responsible for conducting monitoring visits told  us that they were instructed by Education to use state plans as the criteria  for gauging states’ compliance with Reading First reading program  requirements, but that they were provided no formal written guidance or  training.  A senior Education attorney who is currently working with  Reading First program officials told us that he was not aware that they had  used this approach and that he felt that the statutory requirements should  also play an important role in the monitoring process.  Following the  publication of the IG’s report in September, Education’s Office of General  Counsel has provided training to senior management on internal control  requirements and has begun working with the Reading First office to  develop procedures to guide the department’s activities.
Despite the statutory prohibition against mandating or endorsing curricula  and the department’s stated approach to rely on state plans, and the  processes articulated in them, to assess compliance, states reported to us  several instances in which Reading First officials or contractors appeared  to intervene to influence their selection of reading programs and  assessments. For example, officials from four states reported receiving  suggestions from Education or its contractors to adopt specific reading  programs or assessments. Specifically, two states reported that it was  suggested that they adopt a particular reading assessment. Similarly,  Education’s first IG report also documented one instance in which  Reading First officials at Education worked in concert with state  consultants to ensure that a particular reading program was included on  that state’s list of approved reading programs.
In addition, states reported that Education officials or contractors  suggested that they eliminate specific reading programs or assessments  related to Reading First. Specifically, according to our survey results,  officials from 10 states reported receiving suggestions that they eliminate  specific programs or assessments. In some cases, the same program was  cited by officials from more than one state. In one instance, state officials  reported that Education officials alerted them that expert reviewers  objected to a reading program that was under consideration but not  named explicitly in the state’s application. An official from a different state  reported receiving suggestions from Education officials to eliminate a  certain reading program, adding that Education’s justification was that it  was not aligned with SBRR. In another instance, state officials pointed out  that they had adopted a program that was approved by other states,  according to the procedures in their approved state plan, but were told by  Education officials that it should be removed from their list and that  Education would subsequently take a similar course of action with regard  to those other states as well. Also, Education officials did not always rely  on the criteria found in state plans as the basis for assessing compliance.  We found, for example, one summary letter of findings from a monitoring  report in which Education officials wrote that “Two of the monitored  districts were implementing reading programs that did not appear to be  aligned with scientifically based reading research.” Officials we spoke to in  that state told us that they did not feel that they had been assessed on the  basis of the procedures outlined in the state’s plan, but rather that the  reading program itself was being called into question. The IG also found  that Reading First officials communicated to several states against the use  of certain reading programs or assessments, including Rigby and Reading  Recovery.
Officials from a few states also reported being contacted by Education  regarding district Reading First applications or reading programs. For  example, officials from four states reported being contacted by an  Education official about a district application under consideration and one  of those states also reported being approached by staff from one of the  regional technical assistance centers or another contractor for the same  reason. Officials from each of these states indicated that the reason they  were contacted stemmed from the reading programs being used by the  districts in question. In a few cases, state officials reported being  contacted by Education officials regarding the state’s acceptance of a  reading program or assessment that was not in compliance with Reading  First. In one instance, state officials reported that Education contacted  them outside of the normal monitoring process after they had obtained  information from a national Reading First database maintained by a non- profit research organization that districts in the state were using a specific  reading program.
Five states also reported receiving recommendations from Reading First  officials or contractors to change some of the professional development  providers proposed in their original grant applications. When asked about  the specific providers identified for elimination, three of the states  indicated that the providers identified for elimination were in-state  experts. In one case, a state was told that the review panel cited a lack of  detail about the qualifications of the state’s proposed professional  development consultants.
We also found that while Education officials laid out an ambitious plan to  annually monitor every state, they failed to develop written procedures  guiding its monitoring visits. For example, Education did not establish  timelines for submitting final reports to states following monitoring visits,  specifically how and when state officials were expected to follow up with  Education officials regarding findings from the monitoring visits. As a  result, states did not always understand monitoring response procedures,  timelines, and expectations. While we found that most state officials we  spoke with understood that they were to be monitored with the use of  their state plans as the criteria, they did not always understand what was  required of them when responding to monitoring findings. For example,  one state official reported being unaware that the state was supposed to  respond to Education officials about findings from its monitoring report.  An official from another state maintained that he/she was unclear about  the process the state was to follow to respond to findings, and that no  timeline for responding was provided to him/her. Furthermore, one state  reported that findings were not delivered in a timely manner, and another  state reported that Education did not address the state’s responses to the  monitoring findings. Key aspects of an effective monitoring program  include communicating to individuals responsible for the function any  deficiencies found during the monitoring.
section_title: Conclusions
paragraphs: The Reading First program, according to state coordinators, has brought  about changes and improvements to the way teachers, administrators, and  other education professionals approach reading instruction for children in  at-risk, low-performing schools during the critical years between  kindergarten and third grade. To assist states in implementing this large,  new federal reading initiative, Education has provided a wide range of  guidance, assistance, and oversight, that, for the most part, states have  found helpful. However, Education failed to develop comprehensive  written guidance and procedures to ensure that its interactions with states  complied with statutory provisions. Specifically, Education lacked an  adequate set of controls to ensure that Reading First’s requirements were  followed, while at the same time ensuring that it did not intervene into  state and local curricular decisions. We concur with the Education IG’s  recommendations that the Department develop a set of internal  procedures to ensure that federal statutes and regulations are followed  and we feel it is important for the Secretary to follow up on these  recommendations to ensure that they are properly implemented.   Additionally, we feel it is important for the department to have clear  procedures in place to guide departmental officials in their dealings with  state and local officials.  While Education’s stated approach was to rely on  state plans as its criteria for enforcing Reading First’s requirements, states  reported several instances in which it appears that Education officials did  attempt to direct or endorse state and local curricular decisions. Such  actions would prevent states from exercising their full authority under the  law and would violate current statutory restrictions. Balancing Reading  First’s requirements and the limits placed on the department requires  Education to have clear, explicit, and well-documented procedures to  guide its interactions with the states. Failure to do so places the  department at risk of violating the law and leaves it vulnerable to  allegations of favoritism.
Additionally, while Education’s annual monitoring effort for Reading First  is ambitious, it did not provide clear guidelines and procedures to states.  As a result, states were not always aware of their roles and responsibilities  in responding to findings of non-compliance, and Education was not  always consistent in its procedures to follow up with states to resolve  findings and let states know if they had taken proper actions. Key aspects  of an effective monitoring program include transparency and consistency.  Letting all states know in a timely manner whether or not their plans to  address deficiencies are adequate is important to ensure that findings are  dealt with in an appropriate, timely and clear manner.
section_title: Recommendations
paragraphs: In addition to addressing the IG’s recommendations to develop internal   (1) policies and procedures to guide program managers on when to solicit  advice from General Counsel and (2) guidance on the prohibitions  imposed by section 103(b) of the DEOA, we recommend that, in order to  ensure that the department complies with statutory prohibitions against  directing, mandating, or endorsing state and local curricular decisions, the  Secretary of Education also establish control procedures to guide  departmental officials and contractors in their interactions with states,  districts, and schools.
In addition, to help the department conduct effective monitoring of the  Reading First program, we recommend that the Secretary of Education  establish and disseminate clear procedures governing the Reading First  monitoring process. In particular, Education should delineate states’ rights  and responsibilities and establish timelines and procedures for addressing  findings.
section_title: Agency Comments
paragraphs: We provided a draft of this report to the Department of Education and  received written comments from the agency.  In its comments, included as  appendix III of this report, Education agreed with our recommendations  and indicated that it will take actions to address them.
Specifically, Education said it will provide written guidance to all  departmental staff to remind them of the importance of impartiality in  carrying out their duties and not construing program statutes to authorize  the department to mandate, direct, or control curriculum and instruction,  except to the extent authorized by law.  On February 7, 2007, the Secretary  of Education issued a memorandum to senior officers reminding them that  it is important to maintain objectivity, fairness, and professionalism when  carrying out their duties.  The Secretary’s memorandum also emphasizes  the importance of adhering to the statutory prohibitions against  mandating, directing, and controlling curriculum and instruction, and  strongly encourages managers to consult with Education’s Office of  General Counsel early on to identify and resolve potential legal issues.
Also, according to Education’s written comments on our draft report and  the Secretary’s February 7, 2007, memorandum to senior officers, annual  training will be required on internal controls and this training will address  statutory prohibitions against mandating, directing or controlling local  curriculum and instruction decisions.
Regarding its monitoring process for Reading First, in its comments,  Education said that it will develop and disseminate guidelines to states  outlining the goals and purposes of its monitoring efforts, revise the  monitoring protocols, and develop timelines and procedures on states’  rights and responsibilities for addressing monitoring findings.  Education  also included in its response a summary of its actions and planned actions  to address recommendations from the department’s Office of Inspector  General’s recent report on the implementation of the Reading First  program.
As agreed with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days after its  issue date. At that time, we will send copies of this report to appropriate  congressional committees, the Secretary of Education, and other  interested parties. Copies will also be made available upon request. In  addition, the report will be available at no charge on GAO’s Web site at  http://www.gao.gov. If you or your staff have any questions about the  report, please contact me at (202) 512-7215. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objective was to answer the following questions: (1) What changes  have occurred to reading instruction since the inception of Reading First?  (2) What criteria have states used to award Reading First sub-grants to  districts, and what, if any, difficulty did states face in implementing the  program? (3) What guidance, assistance, and oversight did Education  provide states related to the Reading First program? To answer these  questions, we collected both qualitative and quantitative information about  the Reading First program from a variety of sources. We conducted a Web- based survey of the Reading First Directors in all 50 states and the District  of Columbia. We also obtained and analyzed data from the Department of  Education for each state on Reading First districts’ eligibility, applications  and awards for states’ first school year of funding. The first school year of  funding varied across states. Twenty-five states received their first year of  funding in the 2002-2003 school year. Twenty-five states received their first  year of funding in the 2003-2004 school year. To assess the reliability of  this data, we talked to agency officials about data quality control  procedures and reviewed relevant documentation. We excluded two states  because of reporting inconsistencies, but determined that the data for the  other states were sufficiently reliable for the purposes of this report. We  also conducted semi-structured follow-up interviews with Reading First  Directors in 12 states, mostly over the telephone. We conducted site visits  to 4 of the 12 states. During the site visits, we met with state officials, local  program administrators, and state-level technical assistance providers, as  well as school officials from individual schools, including teachers,  principals, and Reading First coaches. In identifying local sub-grant  recipients to meet with in each state, we sought to incorporate the  perspectives of urban, rural, and suburban school districts. We selected  the 12 states to have diversity in a variety of factors, including geographic  distribution, grant size, poverty rate, percentage of students reading at or  below grade level, urban and rural distinctions, the presence of a  statewide list of approved reading programs, and whether states had  reported that they received guidance from Education officials advocating  for or against particular reading programs or assessments. For both the  survey and follow-up interviews, to encourage candid responses, we  promised to provide confidentiality. As a result, state survey responses  will be provided primarily in summary form or credited to unnamed states,  and the states selected for follow-up interviews will not be specifically  identified. Furthermore, in order to adequately protect state identities, we  are unable to provide the names of particular reading programs or  assessments Education officials or contractors suggested a state use or  not use. We did not attempt to verify allegations made by state or local  officials in their survey responses or during interviews or otherwise make  any factual findings about Education’s conduct.
We also visited or talked with administrators from each of the three  regional Reading First Technical Assistance Centers, located in Florida,  Texas and Oregon, as well as RMC Research, the federal contractor tasked  with administering the contract with the technical assistance centers. We  also interviewed several publishers and other providers of reading  curricula and assessments, to obtain their views about changes Reading  First has prompted in states, districts, and schools. We chose these  providers to reflect the perspectives of large, commercial reading textbook  programs that are widely represented nationwide on states’ lists of  approved programs, as well as some other selected providers of reading  curricula, including some that have filed complaints related to Reading  First. We also interviewed Education officials about the implementation of  the Reading First program.
To obtain a better understanding of state program structure, as well as the  nature of interactions between Education officials and state grantees, we  reviewed state grant files, monitoring reports, and related correspondence  for the 12 states where we conducted follow-up interviews. In addition, we  reviewed NCLBA language authorizing Reading First, as well as statements  of work articulating the responsibilities of the regional technical  assistance centers and the contractor tasked with providing assistance to  states in conducting local sub-grant competitions. We conducted our work  from December 2005 through January 2007 in accordance with generally  accepted government auditing standards.
subsections: section_title: Survey of States
paragraphs: To better understand state implementation of the Reading First program,  we designed and administered a Web-based survey of the Reading First  Directors in all 50 states and the District of Columbia. The survey was  conducted between June and July 2006 with 100 percent of state Reading  First Directors responding. The survey included questions about  curriculum; professional development; and state Reading First grant  eligibility, application, award, and implementation processes. The survey  contained both closed- and open-ended questions. For the open-ended  questions, we used content analysis to classify and code the responses  from the states such as the publishers on states’ approved lists. We had  two people independently code the material, then reconciled any  differences in coding. Because this was not a sample survey, there are no  sampling errors. However, the practical difficulties of conducting any  survey may introduce nonsampling errors, such as variations in how  respondents interpret questions and their willingness to offer accurate  responses. We took steps to minimize nonsampling errors, including pre- testing draft instruments and using a Web-based administration system.  Specifically, during survey development, we pre-tested draft instruments  with one expert reviewer and Reading First Directors in four states during  April and May 2006. In the pre-tests, we were generally interested in the  clarity of the questions and the flow and layout of the survey. For example,  we wanted to ensure definitions used in the survey were clear and known  to the respondents, categories provided in closed-ended questions were  complete and exclusive, and the ordering of survey sections and the  questions within each section was appropriate. On the basis of the pre- tests, the Web instrument underwent some slight revisions. A second step  we took to minimize nonsampling errors was using a Web-based survey.  By allowing respondents to enter their responses directly into an  electronic instrument, this method automatically created a record for each  respondent in a data file and eliminated the need for and the errors (and  costs) associated with a manual data entry process. To further minimize  errors, programs used to analyze the survey data were independently  verified to ensure the accuracy of this work.
section_title: Appendix II: Descriptions of Assessment Types
subsections: section_title: Reading First Assessment Requirements and Descriptions
paragraphs: Reading programs under Reading First must include rigorous assessments with proven validity and reliability.
Assessments must measure progress in the five essential components of reading instruction and identify students who may be at  risk for reading failure or who are already experiencing reading difficulty.
Reading programs under Reading First must include screening assessments, diagnostic assessments, and classroom-based  instructional assessments of progress.
section_title: Appendix III: Comments from the Department of Education
section_title: Appendix IV: GAO Contacts and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: Bryon Gordon, Assistant Director, and Tiffany Boiman, Analyst-in-Charge,  managed this engagement and made significant contributions to all  aspects of this report. Sonya Phillips, Sheranda Campbell, Janice Ceperich,  and Andrew Huddleston also made significant contributions. Jean  McSween provided methodological expertise and assistance. Sheila  McCoy and Richard Burkard delivered legal counsel and analysis.  Susannah Compton, Charlie Willson, and Scott Heacock assisted with  message and report development.