+-----------------------------------------------------------------------+
| The Department's Administration of Selected                           |
|                                                                       |
| Aspects of the Reading First Program                                  |
+-----------------------------------------------------------------------+

## FINAL AUDIT REPORT

![](media/image1.wmf)

**ED-OIG/A03G0006**

**February 2007**

+-------------------------+----------------------+---------------------+
| Our mission is to       |                      | U.S Department of   |
| promote the efficiency, |                      | Education           |
| effectiveness, and      |                      |                     |
| integrity of the        |                      | Office of Inspector |
| Department's Programs   |                      | General             |
| and Operations.         |                      |                     |
|                         |                      | Philadelphia, PA    |
+-------------------------+----------------------+---------------------+

NOTICE

> Statements that managerial practices need improvements, as well as
> other conclusions and recommendations in this report, represent the
> opinions of the Office of Inspector General. Determinations of
> corrective action to be taken will be made by the appropriate
> Department of Education officials.
>
> In accordance with Freedom of Information Act (5 U.S.C. § 552),
> reports issued by the Office of Inspector General are available to
> members of the press and general public to the extent information
> contained therein is not subject to exemptions in the Act.

##  {#section .Style1}

![](media/image2.wmf){width="1.125in" height="1.125in"}

THE INSPECTOR GENERAL

February 22, 2007

#####  {#section-1 .Enclosure-Number}

##### Memorandum {#memorandum .Enclosure-Number}

**TO:** Ray Simon

Deputy Secretary

Lead Action Official

Lawrence A. Warder

Chief Financial Officer

Office of the Chief Financial Officer

**FROM:** John P. Higgins, Jr. /s/

**SUBJECT:** Final Audit Report

> The Department's Administration of Selected Aspects of the Reading
> First Program

Control Number ED-OIG/A03G0006

Attached is the subject final audit report that covers the results of
our review of selected aspects of the Department's administration of the
Reading First program during the period August 1, 2001, through
September 30, 2004. An electronic copy has been provided to your Audit
Liaison Officers. We received your comments, which concurred with the
recommendations, but only concurred in part with the findings in our
draft report.

Corrective actions proposed (resolution phase) and implemented (closure
phase) by your offices will be monitored and tracked through the
Department's Audit Accountability and Resolution Tracking System
(AARTS). ED policy requires that you enter your final corrective action
plan (CAP) for our review in the automated system within 30 days of the
issuance of this report.

In accordance with the Inspector General Act of 1978, as amended, the
Office of Inspector General is required to report to Congress twice a
year on the audits that remain unresolved after six months from the date
of issuance.

In accordance with the Freedom of Information Act (5 U.S.C. §552),
reports issued by the Office of Inspector General are available to
members of the press and general public to the extent information
contained therein is not subject to exemptions in the Act.

400 MARYLAND AVE., S.W. WASHINGTON, D.C. 20202-1510

We appreciate the cooperation given us during this review. If you have
any questions, please call Bernard Tadley, Regional Inspector General
for Audit, at 215-656-6279.

Attachment

cc: Anne Campbell, Chief of Staff, Office of Elementary and Secondary
Education

electronic cc: Dolores Warner, Audit Liaison Officer, Office of
Elementary and Secondary Education

Gail Cornish, Audit Liaison Officer, Office of the Chief Financial
Officer

TABLE OF CONTENTS

####### Page

##### EXECUTIVE SUMMARY................................................................................ 1

##  {#section-2 .Style1}

##### BACKGROUND.............................................\................................................ 3

**AUDIT
RESULTS.........................................................\..................................
6**

**FINDING NO. 1 -- Sessions at The Secretary's Reading Leadership**

**Academies Focused on a Select Number of Reading**

**Programs....................................................................
6**

**FINDING NO. 2 -- The Secretary's Reading Leadership Academy Handbook**

**and Guidebook Appeared to Promote the Dynamic**

**Indicators of Basic Early Literacy Skills Assessment**

#####  Test..........................................................................12

**FINDING NO. 3 -- The Department Did Not Adequately Assess Issues**

**of Bias and Lack of
Objectivity.......................................14**

**OTHER
MATTERS.....................................................................................
17**

**OBJECTIVE, SCOPE, AND
METHODOLOGY................................................. 19**

**ATTACHMENT 1 -- The Secretary's Reading Leadership Academy 1, Day 2,**

**Participant Evaluations**

**ATTACHMENT 2 -- The Secretary's Reading Leadership Academy 2, Day 2,**

**Participant Evaluations**

**ATTACHMENT 3 -- The Secretary's Reading Leadership Academy 3, Day 2,**

**Participant Evaluations**

**ATTACHMENT 4 -- Auditee's Comments**

EXECUTIVE SUMMARY

As part of the U.S. Department of Education\'s (Department) efforts to
equip states with the information and resources needed to implement the
Reading First provision of the No Child Left Behind Act of 2001 (NCLB
Act), the Department and the National Institute for Literacy (NIFL)
sponsored three major reading academies, the Secretary's Reading
Leadership Academies (RLAs). The RLAs were held in Washington, D.C., in
January and February 2002, and hosted policymakers and key education
leaders from every state and territory in the nation. The academies were
designed to help state leaders gear up for the implementation of Reading
First, the Department's program to improve the quality of reading
instruction in kindergarten through third grade. The Department also
provided support to states and districts in their Reading First program
implementation by funding the National Center for Reading First
Technical Assistance (NCRFTA) contract.

The objective of our audit was to determine whether the Department
carried out its role in accordance with applicable laws and regulations
in administering the RLAs and related meetings and conferences, the
NCRFTA contract award process, and its website and guidance for the
Reading First program.[^1]

Our audit disclosed that the Department generally administered its
Reading First website, and its *Guidance for the Reading First Program*,
dated April 2002, in accordance with applicable laws and regulations.
With regard to the RLAs, we concluded that the Department did not have
controls in place to ensure compliance with the Department of Education
Organization Act (DEOA) and NCLB Act curriculum provisions.
Specifically, we found that: 1) the "Theory to Practice" sessions at the
RLAs focused on a select number of reading programs; and 2) the RLA
Handbook and Guidebook appeared to promote the Dynamic Indicators of
Basic Early Literacy Skills (DIBELS) Assessment Test. With regard to RMC
Research Corporation's (RMC) technical proposal for the NCRFTA contract,
we concluded that the Department did not adequately assess issues of
bias and lack of objectivity when approving individuals to be technical
assistance providers before and after the NCRFTA contract was awarded.

We recommend that the Assistant Secretary for Elementary and Secondary
Education --

-   Establish controls to ensure compliance with, and avoid the
    appearance of violating the DEOA and the NCLB Act curriculum
    provisions, especially when organizing conferences where specific
    programs of instruction are likely to be formally discussed or
    presented at Department sponsored events;

```{=html}
<!-- -->
```
-   Establish controls to ensure it does not promote curriculum or
    create the appearance that it is endorsing or approving curriculum
    in its conference materials and related publications; and

-   In coordination with the Chief Financial Officer, establish controls
    to ensure adequate assessments of bias and lack of objectivity for
    individuals proposed to perform Department contract work are
    conducted by the Department and its contractors.

The Department concurred with the recommendations in the draft report
and provided proposed corrective actions to address each recommendation.
However, it only agreed in part with the findings in the report. The
Department stated that the report did not present a balanced summary of
the activities reviewed, asserting that the report did not recognize the
positive aspects of the activities and challenges faced in planning
these activities (the RLAs). We acknowledge that there were positive
aspects of the RLAs; however, the purpose of this report is to identify
opportunities for improvement. To address some of the Department's
concerns, we included the RLA Participant Evaluations for day two of the
second RLA in this final report (see Attachment 2), so that all of the
participants' comments (positive and negative) are included in the
report. The Department's specific comments are addressed after each
finding, and the full text of its response is provided as Attachment 4
to this report.

We also found that there is interest in placing more emphasis on a
reading program's scientific evidence of effectiveness in determining
its eligibility for Reading First funds. We suggest that the Department
and Congress, during the next reauthorization of the law, clarify
whether reading programs need to have scientific evidence of
effectiveness in order to be eligible for funding under Reading First.
This issue is discussed in the Other Matters section of the report.

BACKGROUND

In 2000, the Partnership for Reading, now authorized by the Elementary
and Secondary Education Act (ESEA), as amended by the NCLB Act (Public
Law 107-110), was established. The Partnership was a collaborative
effort by three federal agencies: NIFL, the National Institute of Child
Health and Human Development (NICHHD), and the Department; to bring the
findings of evidence-based reading research to the educational
community, families, and others with an interest in helping all people
learn to read well. According to NIFL, the Partnership\'s mission was to
disseminate evidence-based research, a focus that made it substantively
different from earlier information dissemination efforts and
clearinghouses. This mandate to use evidence-based research as the basis
for making decisions about reading instruction was continued by the work
of the National Reading Panel (NRP), assigned by Congress in 1997 to
review the available research. The NRP examined more than 460 studies to
extract the essential findings about what has been scientifically proven
to work in reading instruction.

The ESEA, as amended by the NCLB Act on January 8, 2002, established the
Reading First program. Title 1, Part B, Section 1002(b)(1) of the ESEA
authorized an appropriation for Reading First of \$900,000,000 for
fiscal year 2002 and "sums as may be necessary for each of the five
succeeding fiscal years." The appropriations for fiscal years 2003,
2004, 2005, and 2006 were \$993,500,000, \$1,023,923,000,
\$1,041,600,000, and \$1,029,234,000, respectively. From January 2002,
through September 2006, two Department officials in the Department's
Office of Elementary and Secondary Education (OESE) administered Reading
First, a Reading First Director and an Education Program Specialist.

In April 2002, the Department issued guidance for the Reading First
program and placed this guidance on the Department's website. The
website provided information about Reading First for teachers,
principals, parents, state and local education officials, and anyone
with an interest in improving reading instruction and increasing student
achievement. According to the Reading First guidance, the Reading First
program focused on putting proven methods of early reading instruction
in classrooms. Through Reading First, states and districts received
support to apply scientifically based reading research (SBRR)---and the
proven instructional and assessment tools consistent with this
research---to ensure that all children learned to read well by the end
of third grade. The Department provided formula grants to states that
submitted an approved application. State Educational Agencies (SEAs)
then awarded subgrants to eligible Local Educational Agencies (LEAs) on
a competitive basis. SEAs funded subgrants that showed the most promise
for raising student achievement and for successful implementation of
reading instruction, particularly at the classroom level. Only programs
that were based on SBRR were eligible for funding through Reading First.

Title 1, Part B, Section 1208(6) of the ESEA defines SBRR as research
that:

A.  Applies rigorous, systematic, and objective procedures to obtain
    valid knowledge relevant to reading development, reading
    instruction, and reading difficulties; and

B.  Includes research that-

```{=html}
<!-- -->
```
i.  Employs systematic, empirical methods that draw on observation or
    experiment;

ii. Involves rigorous data analyses that are adequate to test the stated
    hypotheses and justify the general conclusions drawn;

iii. Relies on measurements or observational methods that provide valid
     data across evaluators and observers and across multiple
     measurements and observations; and

iv. Has been accepted by a peer-reviewed journal or approved by a panel
    of independent experts through a comparably rigorous, objective, and
    scientific review.

NIFL, an interagency group composed of the Secretaries of Education,
Labor, and Health and Human Services (HHS), was authorized under the
NCLB Act to help children, youth, and adults learn to read by supporting
and disseminating SBRR. In 1998-1999, NIFL was appropriated \$5 million
in funds under the Reading Excellence Act (REA) program to conduct a
National Reading Research Dissemination Campaign (NRRDC).[^2] In
September 2000, NIFL contracted with RMC to perform tasks for the NRRDC.

In August 2001, five months prior to the Reading First program
legislation being signed into law, the Department started gearing up its
Reading First program and wanted to disseminate the findings about SBRR
to the states. The Department did not have the funding to hold
conferences on SBRR so it requested NIFL's help.

In September 2001, the RLAs were added as a task under NIFL's NRRDC
contract with RMC. The Department and NIFL sponsored three reading
academies, the Secretary's RLAs, which were held in Washington, D.C., on
January 23-25, February 13-15, and February 20-22, 2002. In planning the
RLAs, the Assistant Secretary (former) for OESE and other Department
officials worked with NIFL, RMC, and numerous individuals outside of the
Department. Several of these individuals were associated with the Direct
Instruction program. The RLAs featured presentations on topics, such as
effective reading instruction, the selection of reading programs,
accountability and assessment in reading, professional development for
reading teachers and others, and instructional leadership. Department
officials and researchers from research institutions made presentations
and led discussions with participants on teaching reading based on
scientific research.

The goals of the RLAs were to help states:

-   Develop a knowledge base about scientifically based reading
    instruction;

-   Build capacity to design and sustain professional development for
    teachers in scientifically based reading instruction;

-   Acquire the knowledge and tools to implement the Reading First
    initiative; and

-   Partner with school districts and the Department to improve reading
    achievement.

The RLAs also included a session, titled "Theory to Practice: A Panel of
Practitioners." The speakers discussed how implementing a scientifically
based reading program had brought about great improvements in the
reading skills of their kindergarten through third grade students. In
addition, there was a luncheon speaker on the second day of the first
and third RLA who discussed the improved reading scores of his school's
students since implementing its reading program.[^3]

At each RLA, attendees were provided an RLA Handbook. The book included
copies of the presentations from each RLA session. There were three
different versions of the RLA Handbook, as it was specific to the
particular RLA attended. At the end of each day of each RLA, attendees
were provided with an evaluation form to critique that day's training.
The evaluation forms were provided to the Department.

The Department provided support to states and districts in their Reading
First program implementation by funding the NCRFTA contract. The NCLB
Act authorized the NCRFTA to provide comprehensive technical assistance
to states and districts over five years. The total cost to the
Department for full performance of the contract was estimated to be
\$6,891,035. RMC was the only firm that responded to the solicitation.
RMC was awarded the contract on September 30, 2003, and it currently
serves as the national coordinator for Reading First technical
assistance. The purpose of the contract was to establish three regional
centers located in the western, central, and eastern regions of the
United States. RMC subcontracted with the Institute for the Development
of Educational Achievement (University of Oregon, Eugene), the Texas
Center (Vaughn Gross Center) for Reading and Language Arts (University
of Texas, Austin), and the Florida Center for Reading Research (Florida
State University, Tallahassee) to provide technical assistance to states
and districts in the western, central and eastern regions of the United
States, respectively.

The Department is generally prohibited from exercising control over any
school's curriculum or program of instuction. Section 3403(b) of the
DEOA provides that Department officials shall not construe any provision
of a Department program as authorizing the Department to exercise any
direction, supervision, or control over the curriculum or program of
instruction of any school, or school system. Section 9527(a) of the
ESEA, as amended by the NCLB Act, prohibits Department officials from
construing any provision of the NCLB Act as authorizing such officials
to mandate, direct, or control an SEA, LEA, or school's curriculum, or
program of instruction. Further, Section 9527(b) prohibits the
Department from using any NCLB Act program funds to endorse, approve, or
sanction any curriculum.

A number of complaints have been made that the Department exercised
undue influence in its administration of the Reading First program, by
promoting or endorsing specific reading programs, materials, assessment
instruments, and models of instruction. In addition to this audit, the
OIG conducted an inspection of the Department's administration of the
Reading First grant application process, a series of audits of selected
states' (Wisconsin, New York, and Georgia) implementation of the Reading
First program, as well as an audit of RMC's administration of the
Reading First program contracts.[^4]

AUDIT RESULTS

Our audit disclosed that the Department generally administered its
Reading First website and its *Guidance for the Reading First Program*,
dated April 2002, in accordance with applicable laws and
regulations.[^5] With regard to the RLAs, we concluded that the
Department did not have controls in place to ensure compliance with the
DEOA and NCLB Act curriculum provisions. Specifically, we found that: 1)
the "Theory to Practice" sessions at the RLAs focused on a select number
of reading programs; and 2) the RLA Handbook and Guidebook appeared to
promote the DIBELS Assessment Test. With regard to RMC's technical
proposal for the NCRFTA contract, we concluded that the Department did
not adequately assess issues of bias and lack of objectivity when
approving individuals to be technical assistance providers before and
after the NCRFTA contract was awarded.

The Department concurred with the recommendations in the draft report
and provided proposed corrective actions to address each recommendation.
However, it only agreed in part with the findings in the report. The
Department's specific comments are addressed after each finding, and the
full text of its response is provided as Attachment 4 to this report.

**Finding No. 1 -- Sessions at The Secretary's Reading Leadership
Academies Focused on a Select Number of Reading Programs**

We concluded that the Department did not have controls in place to
ensure compliance with the DEOA and NCLB Act curriculum provisions.[^6]
We found that: 1) only a select number of reading programs were
discussed during the "Theory to Practice: A Panel of Practitioners"
sessions; 2) at the first and third RLAs, the luncheon speaker's
presentation featured one of the few reading programs discussed during
the "Theory to Practice" sessions; and 3) participants at the first and
third RLAs expressed concerns that certain programs were being endorsed
and promoted by the Department.

In January and February 2002, the Department and NIFL sponsored three
Secretary's RLAs in Washington, D.C. According to NIFL, the RLAs were
designed to help state leaders gear up for swift and successful
implementation of Reading First. The RLAs included a session titled
"Theory to Practice: A Panel of Practitioners."

The Director of NIFL informed us that the intent of the RLAs was to
disseminate information on the SBRR findings to the states. The Director
also stated that the RLAs started as a joint effort between the
Department and NIFL. However, the Director stated that after the
Assistant Secretary (former) for OESE proposed that the Secretary of
Education (former) sponsor the RLAs, the Department became much more
involved in the planning of the RLAs and controlled the meetings. The
Department also exercised control over the content and presenters. In
planning the content of the RLAs, the Department worked with numerous
individuals outside of the Department. Several of these individuals were
associated with the Direct Instruction program. The RLAs became known as
the Secretary's RLAs.

# [Composition of the "Theory to Practice" Panels]{.underline}

The Department included only a select number of reading programs for
discussion by the "Theory to Practice" panel members. The following
chart shows the limited number of programs presented by "Theory to
Practice" panels at the RLAs:

+-----------------+-------------------------------------+-------------+
| Reading         | Panel Member - Position at the time | Reading     |
| Leadership      | of the RLA                          | Programs    |
| Academy (RLA)   |                                     |             |
|                 |                                     | Di          |
|                 |                                     | scussed[^7] |
+-----------------+-------------------------------------+-------------+
| RLA 1, Jan.     | Principal, City Springs Elementary, | Direct      |
| 23-25, 2002     | Baltimore, MD                       | Instruction |
+-----------------+-------------------------------------+-------------+
|                 | Reading Facilitator, Parham School, | Direct      |
|                 | Cincinnati, OH                      | Instruction |
+-----------------+-------------------------------------+-------------+
|                 | Principal, Parham School,           | Direct      |
|                 | Cincinnati, OH                      | Instruction |
+-----------------+-------------------------------------+-------------+
|                 | Teacher, Tovashal Elementary        | Open Court  |
|                 | School, Murrieta, CA                |             |
+-----------------+-------------------------------------+-------------+
|                 |                                     |             |
+-----------------+-------------------------------------+-------------+
| RLA 2, Feb.     | Principal, City Springs Elementary, | Direct      |
| 13-15, 2002     | Baltimore, MD                       | Instruction |
+-----------------+-------------------------------------+-------------+
|                 | Asst. Administrator, Washington     | Open        |
|                 | Reads, WA Office of the             | Court,[^8]  |
|                 | Superintendent of Public            | Read Well   |
|                 | Instruction, Olympia, WA            |             |
+-----------------+-------------------------------------+-------------+
|                 | Principal, Weaver Elementary        | Houghton-   |
|                 | School, Weaver, AL                  | Mifflin,^8^ |
|                 |                                     | Soar to     |
|                 |                                     | Success     |
+-----------------+-------------------------------------+-------------+
|                 |                                     |             |
+-----------------+-------------------------------------+-------------+
| RLA 3, Feb.     | Principal, City Springs Elementary, | Direct      |
| 20-22, 2002     | Baltimore, MD                       | Instruction |
+-----------------+-------------------------------------+-------------+
|                 | Principal, Parham School,           | Direct      |
|                 | Cincinnati, OH                      | Instruction |
+-----------------+-------------------------------------+-------------+
|                 | Asst. Superintendent, L.A. Unified  | Open Court  |
|                 | School District, CA                 |             |
+-----------------+-------------------------------------+-------------+

During the "Theory to Practice" sessions, each of the panel members
discussed the reading success stories of the students in his or her
school or district. A question and answer period with the audience
followed. At the first RLA, held on January 23-25, 2002, three of the
four panel members talked about the Direct Instruction program. The
fourth panel member discussed the Open Court program, held up the Open
Court Reader for the audience to view, and read a brief passage from it.

After the "Theory to Practice" session, there were many comments by
attendees indicating they felt the RLA was biased toward Direct
Instruction and Open Court. Consequently, the next day, the Director of
the Reading First Office (former) explained to the audience at the first
RLA that the intention of the panelists was not to "plug" certain
programs. The Department invited some of the same panel members back to
participate on the "Theory to Practice" panels at the second and third
RLAs. However, the Department did not instruct any of the panel members
to refrain from naming the programs that they were using.

At the second RLA, held on February 13-15, 2002, the "Theory to
Practice" panel included three panelists. One of the three panelists was
a participant from the first RLA. Once again, this panelist discussed
the Direct Instruction program.

At the third RLA, held on February 20-22, 2002, the "Theory to Practice"
panel was composed of three panelists. Two of the three panelists had
participated in previous panels and once again they discussed the Direct
Instruction program. The third panel member discussed the Open Court
program and read a series of letters from a teacher, which described how
the teacher was at first skeptical of Open Court, but took the training,
taught it in class, and became a big supporter of the program.

An email, to the Director of the Reading First Office (former), from a
colleague outside of the Department, dated January 15, 2002, with a
subject line of "Fwd: School Using DI in Cinn." suggested the Department
may have intentionally wanted to showcase the Direct Instruction program
by selecting "Theory to Practice" panelists who used Direct Instruction
successfully in their district or at their schools. The email stated --

When \[an RLA consultant\] had me looking for possible administrators to
present at RF \[Reading First\] academies \[the former principal of
Wesley Elementary School in Houston, TX, which has one of the longest
continuous Direct Instruction implementations in the country\]
recommended the principal of this school \[Parham School, Cincinnati,
OH\].

The principal of Parham School was a panelist for two of the three
"Theory to Practice" sessions, and also discussed the Direct Instruction
program.

# [Selection of the Luncheon Speaker]{.underline}

In addition to the "Theory to Practice" session, there was a luncheon
speaker on the second day of each academy. The luncheon speaker at the
first and third RLAs was the principal of the Seed Academy and Harvest
Preparatory School. The principal's presentation focused on one reading
program, Direct Instruction. During both of his presentations, the
principal discussed how he was looking for a model of successful
instruction to implement at his school when he learned of Direct
Instruction. The principal then used slides to show the improved reading
scores of his school's students since implementing Direct Instruction.

An email, dated January 14, 2002, from the Director of the Reading First
Office (former) inviting the principal of the Seed Academy and Harvest
Preparatory School to speak at the RLAs indicated that the Department
may have favored Direct Instruction. The email stated --

We have chatted several times at ADI \[Association for Direct
Instruction\] Eugene conferences, and recently (I think) about the DI
\[Direct Instruction\]-based middle school my wife and I started in
rural Africa several years ago. I am still the President of the Board of
that school . . . . I've also been working with a bunch of public
schools in my home of Baltimore, trying to do what you've done at Seed.

. . . we would hugely appreciate it if you could make time in your
schedule to speak a bit at our never-been-done before Reading Leadership
Academies.

The Government Accountability Office's (GAO) *Standards for Internal
Control in the Federal Government* requires that agencies develop
detailed policies and procedures, and practices to fit their agency's
operations and to ensure that they are built into and are an integral
part of operations. Policies and procedures are a part of the control
activities that enforce management's directives and are an integral part
of an entity's planning, implementing, reviewing, and accountability for
stewardship of government resources and achieving effective results.
Control activities help to ensure that actions are taken to address
risk. As a part of these control activities, policies and procedures
should have been developed and implemented to minimize the risks the
Department may face when it engages external speakers at conferences it
sponsors. One such risk was the risk of the perception that the
Department favored a particular program, tool, or entity. Another risk
was the risk of violating the DEOA, which established the Department and
describes the Federal-State relationship in education as follows --

> Section 3403 (b) No provision of a program administered by the
> Secretary or by any other officer of the Department shall be construed
> to authorize the Secretary or any such officer to exercise any
> direction, supervision, or control over the curriculum, program of
> instruction, administration, or personnel of any educational
> institution, school, or school system . . . over the selection or
> content of . . . textbooks, or other instructional materials by any
> educational institution or school system, except to the extent
> authorized by law.[^9]

Section 9527(b) of the NCLB Act (20 U.S.C. Section 7907) reinforces the
language in the DEOA. Specifically, it prohibits funds provided to the
Department from being used by the Department to endorse, approve, or
sanction any curriculum designed to be used in an elementary or
secondary school.

### RLA Participant Comments

As a result of the Department not having controls to ensure compliance
with the DEOA, and the NCLB Act prohibitions against endorsing or
promoting programs of instruction, some attendees at the RLAs felt that
the Department was endorsing the Direct Instruction and Open Court
reading programs. The comments expressed on the evaluation forms from
the first and third RLAs included[^10] --

-   "The . . . Theory to Practice Panel -- was very poor. It sounded
    like a sales job for a program as opposed to a description of
    enabling teachers to teach reading."

-   "I felt like it was simply a push for a national curriculum. I think
    I'll go buy shares in Open Court!"

-   "Panel was a sales job for Direct Instruction and Open Court."

-   "Please do not promote a program (Open Court) (Direct Instruction).
    This is not the Department of Education's place to do.

```{=html}
<!-- -->
```
-   "I felt like I was in a Direct Instruction sales pitch all day.
    Thanks for including at least one other program."

-   "I felt it was wrong to showcase one specific program (D.I.)
    excessively . . .."

-   "Today's sessions may have given an excessive government endorsement
    to Direct Instruction."

In addition, there was a perception by some states that the programs
discussed at the RLAs were part of a Department "approved list" of
Reading First programs. As a result, in April 2002, the Reading First
Office was compelled to put a notice on its website clarifying that the
Department did not have an approved list of reading programs for use
with Reading First funds.

Nonetheless, the view that there was a Department "approved list" of
Reading First reading programs persisted, and in a letter to Reading
First State Directors, dated October 11, 2005, over two years after
Reading First applications were approved, the Director of the Reading
First Office (former) wrote --

> The U.S. Department of Education has consistently and openly stated
> that there is no Federal "approved list" of reading programs for use
> under the Reading First program. The Department has not developed or
> endorsed any list of instructional programs as appropriate for use,
> nor has the Department indicated a preference for certain kinds of
> instructional programs, such as basal textbook-based programs.

The sessions at the RLAs focused on a select number of reading programs,
which gave attendees the impression that the Department was promoting
these specific reading programs. Since the Department wanted senior
level state leaders, who would be Reading First program policymakers, to
attend the RLAs, it should have taken all action possible to minimize
the perception that it could be viewed as endorsing any reading program.
The Department had control of the agenda and presenters for the RLAs and
should have ensured that nothing in the RLAs could be viewed as
endorsements or approval of any particular program. As one of the
sponsors, the Department had a responsibility to avoid all appearances
of violating the letter and spirit of the DEOA or Section 9527(b) of the
NCLB Act. The appearance that the Department is endorsing or approving
curriculum in its conference sessions can damage the integrity of the
event and the reputation of the Department.

The evidence above calls into question whether the Department violated
the DEOA and the NCLB Act curriculum provisions.

## Recommendation

We recommend that the Assistant Secretary for Elementary and Secondary
Education --

1.  Establish controls to ensure compliance with, and avoid the
    appearance of violating the DEOA and NCLB Act curriculum provisions,
    especially when organizing conferences where specific programs of
    instruction are likely to be formally discussed or presented at
    Department sponsored events.

**Department Comments**

The Department concurred with the recommendation and provided proposed
corrective action to address it. However, the Department agreed only in
part with the finding, stating that the finding failed to provide a
balanced summary of the activities reviewed.

The Department explained --

> While we agree that the audience should have been adequately advised
> that the Department was not promoting or endorsing a specific program,
> we do not think that there is any problem in having panelists
> highlight certain successful programs or identify the specific
> programs they are implementing. . . . To the contrary, we have
> received positive feedback that this is the kind of helpful
> information that applicants and grantees need to make their programs
> more effective. Furthermore, when panelists failed to specifically say
> what program they were implementing, participants during the question
> and answer session asked for the program to be named . . . . Thus, it
> is reasonable for the panelists to mention the names of the programs
> they were using, so that the Department can carry out its important
> statutory responsibilities to disseminate information on effective and
> promising practices.

The Department further stated --

> . . . \[it\] cannot be expected to present information on all possible
> programs. It is reasonable that the Department should be able to
> present information on certain programs, as long as it is made clear
> that the programs presented are merely examples of the types of
> programs that might be supported with Reading First funds and that the
> presentation was not intended to be an endorsement or promotion of a
> specific program.

The Department also stated it was concerned that steps taken by
Department officials to clarify the false impression that was left with
some of the conference attendees were represented in the report as
further evidence of its initial failure to indicate that it was not
endorsing or promoting particular programs. These actions should have
been recognized as positive steps that Department personnel
expeditiously took to clarify this false impression. These actions
included:

-   Informing the audience at the first RLA that the purpose of the
    "Theory to Practice" sessions was not to promote or endorse any
    particular program; and

-   Posting a clarifying note on the Reading First website and sending a
    letter to Reading First State Directors indicating that there was
    not an "approved list" of reading programs that could be funded
    under Reading First.

**OIG Response**

We acknowledge that there were positive comments on the RLA evaluation
forms regarding the "Theory to Practice" sessions, (see Attachments 1,
2, and 3). However, the negative comments should have alerted the
Department that some attendees felt the Department was endorsing or
promoting the Direct Instruction and Open Court reading programs. Prior
to conducting the second and third RLAs, the Department should have done
more to mitigate this perception. The Department was advised about this
perception in the following emails:

-   In an email attachment to the Director of the Reading First Office
    (former), from the facilitator of the first RLA, dated January 27,
    2002, titled "Notes from Debriefing of the First RLA," the notes
    include the following point: "Too much emphasis on Direct
    Instruction."

-   Another email, dated February 4, 2002, to the Director of the
    Reading First Office (former), from the President of RMC, included
    feedback from a consultant who was tasked with analyzing the RLA
    participant evaluations from the first RLA. It stated, "And, as
    everyone knows, Open Court and Direct Instruction can't be the only
    shows in town."

Although, additional reading programs were discussed during the second
RLA's "Theory to Practice" session, as stated in the finding, for the
third RLA the Department again only selected panelists that used the
Direct Instruction and Open Court reading programs. This again gave the
impression that the Department was endorsing or promoting these
programs, and that they were approved Reading First programs. Our
concern is that only a few specific reading programs were discussed when
there were other reading programs available.

We do not believe the actions taken by the Department to clarify that it
was not endorsing or promoting any particular programs have been
misrepresented in the audit finding. As stated in the finding, the
perception of an "approved list" still persisted well after the RLAs.
Although OESE put a note on its website in 2002 to address the
perception, it was still compelled to take further action to address the
perception in 2005, over three years after the RLAs were held. This is
evidence that in planning for the first RLA, the Department did not
address the risk of appearing to promote or endorse a particular
program. Further, once it became evident that there was a perception by
some of the attendees of the first and third RLAs that the Department
was endorsing or promoting reading programs, not enough was done to
mitigate this perception.

**Finding No. 2 - The Secretary's Reading Leadership Academy Handbook
and Guidebook, Appeared to Promote the Dynamic Indicators of Basic Early
Literacy Skills Assessment Test**

The Reading First statute required the use of screening, diagnostic, and
classroom-based reading assessments so that teachers could effectively
screen, identify, and overcome reading barriers facing their students.
We found that the Department appeared to promote DIBELS by including an
article featuring it in the RLA Handbook and Guidebook.

At each RLA, attendees were given a RLA Handbook that contained the
PowerPoint slides for each speaker's presentations. The RLA Handbook
also included an article titled, "The Importance and Decision-Making
Utility of a Continuum of Fluency-Based Indicators of Foundational
Reading Skills for Third-Grade High-Stakes Outcomes." This 29-page
article featured DIBELS, one of many screening tools on the market that
could have been used for performing Reading First assessments. The
Department also included this article in the RLA Guidebook. The RLA
Guidebook, published in September 2002, was a collection of the
presentations and resources that formed the basis of the RLAs. While
numerous assessment instruments were listed in the RLA Handbook and
Guidebook, only DIBELS was featured in an article in both books.[^11]

The RLA Handbook was provided at the RLAs, which were held to assist
states in gearing up for their implementation of the Reading First
program. The RLA Guidebook was provided shortly thereafter, when states
were preparing their Reading First applications. As a result, states may
have been given the impression that DIBELS was the assessment test the
Department preferred for use in the Reading First program. In fact, 43
states indicated in their Reading First applications that they intended
to use DIBELS as one of their assessment test instruments.

As cited in Finding 1, federal laws and regulations applicable to this
finding are in the DEOA, and the NCLB Act, which contain provisions
prohibiting the Department from exercising direction or control over
curriculum, or from endorsing any curriculum or program of
instruction.[^12] Also as cited in Finding 1, GAO's *Standards for
Internal Control in the Federal Government* requires that agencies
develop detailed policies and procedures, and practices to fit their
agency's operations and to ensure that they are built into and are an
integral part of operations. Policies and procedures are a part of the
control activities that enforce management's directives and are an
integral part of an entity's planning, implementing, reviewing, and
accountability for stewardship of government resources and achieving
effective results. Control activities help to ensure that actions are
taken to address risk. As a part of these control activities, policies
and procedures should have been developed and implemented to minimize
the risks the Department may face when it provides guidance to
state-level administrators and policy makers. One such risk was the risk
of the perception that the Department favored a particular program,
tool, or entity.

We concluded that the Department did not have controls in place to
ensure it was not promoting curriculum or creating the appearance that
it was endorsing or approving curriculum in its conference materials and
related publications.[^13]

The appearance that the Department is endorsing or approving curriculum
in its conference materials and related publications can damage the
integrity of the event and the reputation of the Department.

### **Recommendation**

We recommend that the Assistant Secretary for Elementary and Secondary
Education --

2.1 Establish controls to ensure it does not promote curriculum or
create the appearance that it is endorsing or approving curriculum in
its conference materials and related publications.

**Department Comments**

The Department concurred with the recommendation and provided proposed
corrective action to address it. The Department did not provide any
specific comments in its response indicating agreement or disagreement
with this finding.

Finding No. 3 -- The Department Did Not Adequately Assess Issues of Bias
and Lack of Objectivity

We found that the Department did not adequately assess issues of bias
and lack of objectivity when approving individuals to be technical
assistance providers before and after the NCRFTA contract was awarded.
Specifically, the Department did not: 1) adequately vet proposed
technical assistance providers resumes, and 2) follow up on reading
related contracts held by technical assistance providers in order to
determine whether views and positions taken could be largely motivated
from the close identification or association of an individual with a
particular point of view or the positions or perspectives of a
particular group.

When RMC submitted its proposal for the NCRFTA contract, it included the
names and resumes of key personnel, including Regional Directors.
Although issues of bias and lack of objectivity of some proposed key
personnel existed, the Department did not address this issue. For
example, the current and former directors of the Western Regional
Technical Assistance Center had associations with Prentice Hall, which
included Prentice Hall publishing a reading text authored by these
individuals in 1997.[^14]

In addition, as a part of the NCRFTA contract negotiations, the
Department requested RMC provide a list of all reading related contracts
held by the staff of each technical assistance center. The following
contracts were reported to RMC: the Central Regional Technical
Assistance Center's former director had been a paid consultant to the
reading program Voyager prior to her work with the Center; and the
Eastern Regional Technical Assistance Center's current director has a
current publishing contract for reading intervention materials with the
publisher SRA/McGraw-Hill. The Department did not follow up on these
issues.

The Department's main reason for reviewing technical assistance
providers' resumes was to determine whether the proposed individuals had
sufficient SBRR expertise and\\or were being paid with Reading First
funds for performing other work related to Reading First. The Department
did not consider whether the proposed individuals had associations with
reading program publishers and or reading programs. In addition, the
Department did not require RMC to vet consultant resumes for issues of
bias and lack of objectivity.

We concluded that the Department did not have controls to ensure
potential sources of bias and lack of objectivity were adequately
assessed. In addition, the Department did not consider associations with
reading program publishers as a potential source of bias because
officials thought that it would limit the pool of technical assistance
providers with expertise in SBRR. Consequently, appearances of bias and
lack of objectivity contributed to the complaints surrounding the
administration of the Reading First program and led to the perception
that some individuals may have been promoting the reading products they
were associated with and may have influenced the products that were
being selected by SEAs and LEAs.

To respond to the ongoing perception of the appearances of bias and lack
of objectivity, in a letter sent to RMC, dated October 11, 2005, two
years after the NCRFTA contract was awarded, the Reading First Director
(former) wrote --

> In providing this assistance, \[technical assistance\], it is
> essential that the National Center avoid all conflicts of interest
> among employees, subcontractors, and consultants who may have
> connections to particular instructional programs or materials used
> under Reading First. Although I understand that the National Center
> does not recommend programs or materials, it is important that steps
> are taken to avoid the appearance of conflicts of interest in this
> area. Individuals associated with the National Center or the three
> regional centers must recuse themselves from participating in
> technical assistance activities that appear to benefit commercial
> entities with whom they are personally connected.

As cited in Finding 1, federal laws and regulations applicable to this
finding are in the DEOA, and the NCLB Act, which contain provisions
prohibiting the Department from exercising direction or control over
curriculum, or from endorsing any curriculum or program of instruction.
Also as cited in Finding 1, GAO's *Standards for Internal Control in the
Federal Government* requires that agencies develop detailed policies and
procedures, and practices to fit their agency's operations and to ensure
that they are built into and are an integral part of operations.
Policies and procedures are a part of the control activities that
enforce management's directives and are an integral part of an entity's
planning, implementing, reviewing, and accountability for stewardship of
government resources and achieving effective results. Control activities
help to ensure that actions are taken to address risk.

Since the DEOA and the NCLB Act prohibit the Department from endorsing
or promoting curriculum, it should have had a process in place to assess
potential sources of bias and lack of objectivity. Without an adequate
assessment of bias and lack of objectivity for individuals proposed to
perform Department contract work, the Department could be placed in a
situation where the public could reasonably question and perhaps
discount or dismiss the work performed simply because of the existence
of a potential bias. Further, the Department should ensure its
contractors also perform this assessment.

Recommendation

We recommend that the Assistant Secretary for Elementary and Secondary
Education, in coordination with the Chief Financial Officer --

1.  Establish controls to ensure adequate assessments of bias and lack
    of objectivity for individuals proposed to perform Department
    contract work are performed by the Department and its contractors.

**Department Comments**

The Department concurred with the recommendation and provided proposed
corrective actions to address it. The Department's response stated --

> We acknowledge that the Department should have in place procedures to
> assess issues of bias and lack of objectivity on the part of potential
> technical assistance providers. However, the mere fact that certain
> individuals may have expertise with respect to particular programs
> should not preclude them from serving as technical assistance
> providers in any capacity. On the other hand, we acknowledge the
> Department should take steps to ensure that such individuals do not
> provide advice in areas in which they may have a financial conflict of
> interest.

**OIG Response**

The finding does not suggest that an individual should be excluded from
serving as a technical assistance provider due to the mere fact he or
she may have expertise with regard to particular programs. The main
point of the finding, agreed to by the Department, is that the
Department should have had policies and procedures in place and should
have taken steps to assess issues of bias and lack of objectivity.

OTHER MATTERS

In order for a reading program to be used in a state's Reading First
program, it must include explicit and systematic instruction in the five
essential components of reading (phonemic awareness, phonics, vocabulary
development, reading fluency (including oral reading skills), and
reading comprehension strategies), as identified by the statute.
According to the Assistant Secretary (former) for OESE and the Chief
(former) of the Child Development and Behavior Branch (CDBB) within the
NICHHD at the National Institute of Health (NIH), a limited number of
reading programs themselves had been rigorously tested and proven to be
effective when the Reading First program began in 2002. However, since
the initiation of the Reading First legislation, there seems to be some
movement to place more emphasis on the scientific evidence of
effectiveness of reading programs rather than just the inclusion of the
five essential components of reading. For example, a letter from Senator
Richard G. Lugar to Secretary Spellings, dated September 8, 2005, stated
--

> In my view, the Department must do a better job of providing clear
> information to states and districts regarding the definition of
> research based programs. Programs that have been rigorously evaluated
> should be emphasized under Reading First. I would encourage the
> Department to send a mailing to its technical assistance centers and
> States directing them to fully consider the scientific evidence of
> effectiveness for programs, not just program components, in
> considering programs for use under Reading First.
>
> In addition, while a large portion of grants have already been
> awarded, perhaps state Reading First directors should be asked to
> revise criteria for awarding funding to better reflect the above
> definition of research-validated programs and to include competitive
> preference points to applicants proposing to implement
> research-validated programs.

In addition, the language included in the 2006 Labor-HHS-Education
appropriations bill (Public Law 109-103), dated July 2005, appears to
support this point of view. The 2006 Labor-HHS-Education appropriations
bill stated --

The Committee intends for funds available under the Reading First
program to encourage and support the use of reading programs with the
strongest possible scientific evidence of effectiveness. The Committee
urges the Department to provide clear guidance to its technical
assistance centers and the States to: fully consider scientific evidence
of effectiveness in rating programs for use under Reading First;
contemplate expanded lists of allowable programs that include innovative
programs with scientific evidence of effectiveness; when awarding new
grants, consider giving preference to those schools that select programs
with strong, scientific evidence of effectiveness...and allow
comprehensive reading programs that have scientific evidence of
effectiveness to be implemented in full, as they have been researched,
without modification to conform to other models of instruction\...The
Committee notes that Reading First materials decisions are to be made at
the school level, subject to the approval of the State.

This point of view is consistent with an August 2006 statement made to
us, by the Chief (former) of the CDBB within the NICHHD at the NIH.[^15]
The Chief (former) stated that publishers certainly have had the time to
test their programs to determine causal effectiveness. He further stated
that by now, there should be a larger group of programs that have been
tested. With the impending reauthorization of the NCLB Act, he believes
there should be a push towards the "next level." Changes should be made
to tighten up the criteria under which programs are eligible for
funding. He also recommended, in an August 16, 2006, email to us, that
--

Federal funds should only be used for those programs, combination of
programs, instructional strategies, reading methods, hybrid
instructional models (Mathes et al., 2005), that have been found to be
effective using experimental research designs that can determine the
causal impact of the programs, strategies, methods, and/or models on
student learning and achievement in reading. Effectiveness should be
established using the most rigorous experimental designs that provide
the strongest evidence of causal validity with priority placed on
randomized clinical trials (RCT) and regression discontinuity studies.
Quasi-experimental research designs could be utilized when randomization
is not possible but with the recognition that causal validity must be
interpreted with caution. Studies of effectiveness should also address
the issue of intervention fidelity, the reliability and validity of the
measures employed, the extent to which relevant people, settings, and
measurement timings are included in the study, and the extent to which
the effects of the program, strategies, methods, and/or models can be
tested with subgroups within the population under study.

Since the legislation is scheduled for reauthorization in 2007, Congress
has an opportunity to clarify whether reading programs should be funded
on the basis of program effectiveness. Congress will also be able to
determine what it means for a program to be "based on scientific reading
research" and whether this definition is consistent with program
effectiveness. Information obtained and deliberated upon, as part of the
reauthorization process, should enable Congress to make the legislation
more responsive to the needs of children by ensuring that quality
programs are funded with Reading First funds.

We suggest that the Department and Congress, during the next
reauthorization of the law, clarify whether reading programs need to
have scientific evidence of effectiveness in order to be eligible for
funding under Reading First.

OBJECTIVE, SCOPE, AND METHODOLOGY

The objective of our audit was to determine whether the Department
carried out its role in accordance with applicable laws and regulations
in administering the RLAs and related meetings and conferences, the
NCRFTA contract award process, and its website and guidance for the
Reading First program. Our audit was limited to the review of the
Department's administration of the RLAs, the RLA Handbook and Guidebook,
its review of RMC's technical proposal for the NCRFTA contract, its
approval process for NCRFTA contract technical assistance providers, its
website, and its *Guidance for the Reading First Program*, dated April
2002. Our audit covered the period from August 1, 2001, through
September 30, 2004.

> To accomplish our audit objective, we interviewed Department officials
> in the Reading First Program Office and in the Office of General
> Counsel. We interviewed the former Assistant Secretary for Elementary
> and Secondary Education, the former Chief of the CDBB within the
> NICHHD at the NIH, and a former Education and Workforce Committee
> staffer. We also interviewed officials from NIFL, RMC, and persons
> involved in the RLAs, including speakers, panelists from one session
> at the RLAs, entitled, "Theory to Practice: A Panel of Practitioners,"
> and participants. We randomly selected 7 of the 81 participants to
> interview from the first RLA, 5 of the 172 participants from the
> second RLA and 5 of the 165 participants from the third RLA. Our
> sampling objective was to randomly identify individuals who
> participated in the RLAs in order to obtain their opinions on the
> information presented at the RLAs. We also interviewed two reading
> experts familiar with SBRR and the three-tier model.[^16]
>
> Additionally, we reviewed the RLA Handbook, the RLA Guidebook, the
> Department's *Guidance for the Reading First Program*, dated April
> 2002, and the Department's Reading First website. We also reviewed raw
> video footage taken at the RLAs. In addition, we reviewed NCRFTA
> contract documents, and Department correspondence related to the audit
> objective. Finally, we reviewed RLA Participant Evaluation and Comment
> forms for all three days, for all three RLAs.
>
> Review of internal controls was not an objective of our audit.
> However, we identified a lack of internal controls as a cause of the
> deficiencies noted in the report.

Our fieldwork was performed from January 25, 2006, through, September 8,
2006. We conducted an exit conference with the Reading First Program
Office on September 27, 2006.

Our audit was performed in accordance with generally accepted government
auditing standards appropriate to the scope of the review described
above.

[^1]: Refer to the Objective, Scope, and Methodology section of the
    report for the scope of work performed related to our audit
    objective.

[^2]: The REA program pre-dated the Reading First program. The REA
    program was designed to provide children with the readiness skills
    and support they needed to learn to read once they enter school;
    teach every child to read by the end of third grade; and use
    research-based methods to improve the instructional practices of
    teachers and other instructional staff.

[^3]: There was a luncheon speaker from the Department on the second day
    of the second RLA.

[^4]: Inspection Report-ED-OIG/I13F0017, Wisconsin Report-A05G0011, New
    York Report-A02G0002, Georgia Report-A04G0003, RMC Report-A03F0022
    (not issued as of the issuance of this report).

[^5]: Refer to the Objective, Scope, and Methodology section of the
    report for the scope of work performed related to our audit
    objective.

[^6]: The OIG Inspection Report ED-OIG/I13F0017 also identified
    instances where the Department did not have a control environment
    that exemplified management accountability.

[^7]: The panel members named the programs that they were using during
    their presentation and during the question and answer period with
    the exception of the Assistant Administrator for Washington Reads
    and the Principal of Weaver Elementary School. These two panel
    members named the reading programs during the question and answer
    period only.

[^8]: The panel member noted that this program was not working for the
    school he or she discussed.

[^9]: The Department's actions with respect to this provision of the law
    are also discussed in the OIG Inspection Report ED-OIG/I13F0017.

[^10]: We have included the participant evaluation forms for Day 2 of
    RLAs #1, #2, and #3 as Attachments 1, 2, and 3, respectively. We did
    not find comments on the second RLA participant evaluation forms
    indicating that the respondents felt the "Theory to Practice"
    session favored specific reading programs. This may have been
    because only one panel member named the program (Direct Instruction)
    she was using during her presentation.

[^11]: The article included an examination of the decision-making
    utility of the DIBELS benchmark goals in the context of a district
    engaged in a school-wide educational reform effort targeting
    phonological awareness and alphabetic principle skills.

[^12]: The Department has not defined the terms "curriculum" or "program
    of instruction."

[^13]: The OIG Inspection Report ED-OIG/I13F0017 also identified
    instances where the Department did not have a control environment
    that exemplified management accountability.

[^14]: In 2003, Prentice Hall published the fourth edition of this text.

[^15]: The former Chief was a drafter of the Reading First legislation.

[^16]: The Three-Tier Reading Model provides intensive early
    intervention for the students most at risk for reading difficulty.
    It consists of three tiers, or levels, of reading instruction:
    primary, secondary, and tertiary.