GAO

United States General Accounting Office

Report to the Chairman, Subcommittee

on Oversight and Investigations,

Committee on Education and the

Workforce, House of Representatives

March 2000 EDUCATION

DISCRETIONARY

GRANTS

Awards Process Could

Benefit From

Additional

Improvements

GAO/HEHS-00-55

Page 1 GAO/HEHS-00-55 Education Discretionary Grants

Contents

Letter 3

Appendixes Appendix I: Scope and Methodology 22

Appendix II: Analysis of Grantee Characteristics for Specific

Programs 30

Appendix III: Comments From the Department of Education 36

Appendix IV: GAO Contacts and Staff Acknowledgements 42

Tables Table 1: Discretionary Programs Reviewed 7

Table 2: Goals of Education’s Redesign Process and Actions Taken 9

Table 3: Fiscal Year 1998 Discretionary Grant Data for Selected

Programs 23

Table 4: Results of Panel Score Analysis on One Program 27

Table 5: Results of Panel Score Analysis Across Programs 28

Table 6: Number and Percentage of Grantees in Office of Bilingual

Education and Minority Languages Affairs Programs, by

Percentage of LEP Students 30

Table 7: Number and Percentage of Grantees by Percentage of

Children in Poverty 31

Table 8: Distribution of Grantees by Percentage of Children in

Poverty by Urban Status 32

Table 9: Number and Percentage of Grantees by Per-Pupil

Expenditure 33

Table 10: Distribution of Grantees by Per-Pupil Expenditure

Controlling for Urban Status 34

Table 11: Percentage of Grant Applicants by Urban Status 35

Abbreviations

CMSA consolidated metropolitan statistical area

GAPS Grant Administration and Payment System

LEA local education agencies

LEP limited English proficient

MSA metropolitan statistical area

OBEMLA Office of Bilingual Education and Minority Languages Affairs

OMB Office of Management and Budget
Page 2 GAO/HEHS-00-55 Education Discretionary Grants
Page 3 GAO/HEHS-00-55 Education Discretionary Grants

United States General Accounting Office

Washington, D.C. 20548

Health, Education, and

Human Services Division

B-282710 Letr

March 30, 2000

The Honorable Peter Hoekstra

Chairman, Subcommittee on

Oversight and Investigations

Committee on Education and

the Workforce

House of Representatives

Dear Mr. Chairman:

The Department of Education (Education) annually provides billions of

dollars in grants to state and local education agencies, school districts,

colleges and universities, and other organizations to conduct various

program and research activities. These funds are distributed

noncompetitively, through formula grants based on certain applicant

characteristics; and competitively, through discretionary grants based on

the Secretary’s judgment. In fiscal year 1999, the Congress appropriated

about $1.7 billion for use in 88 discretionary grant programs that made new

awards during the fiscal year. Education uses the results of peer reviews, in

which a set of experts evaluates the merit of applications, to assist in

making awards decisions.

In the past, applicants expressed concern that the awards process was

time-consuming and burdensome. According to a September 1993 National

Performance Review report, Education estimated that the awards process

required 26 weeks, with only 6 to 8 weeks of that time available for

applicants to prepare applications.1 Applicants expressed frustration with

the limited time allowed to file applications, and grant recipients (grantees)

complained that they had limited time to prepare plans to implement their

projects after receiving their awards. To respond to these concerns, in

December 1995, Education undertook a major effort to totally redesign its

grant awards process. Among other actions, Education decentralized its

grant-making activities, giving authority for this function to its six principal

offices, and streamlined the process. In April 1999, because of your

concern that Education’s grant awards process was still too time1

From Red Tape to Results: Creating a Government That Works Better and Costs Less,

report of the National Performance Review, Vice President Al Gore (Sept. 7, 1993).
B-282710

Page 4 GAO/HEHS-00-55 Education Discretionary Grants

consuming and burdensome, you asked that we provide information on (1)

Education’s progress in implementing the redesigned awards process, (2)

the management controls used to help ensure a fair peer review process

and the costs of peer review, and (3) the extent to which grant awards are

consistent with the results of peer review and the legislation governing

individual grant programs.

To address each objective, we interviewed Education officials and

reviewed agency documents. We reviewed management controls for peer

review using a framework based on potentially vulnerable areas identified

in our previous work.2 We also reviewed Education’s management controls

for the peer review process against government standards that establish

criteria for assessing the effectiveness of such controls. We selected nine

discretionary grant programs that made awards in fiscal year 1998 in four

of Education’s six principal offices to obtain a general sense of how the

peer review process is conducted. These programs accounted for about 15

percent of the agency’s 61 discretionary grant programs that made awards

in fiscal year 1998 and almost 43 percent of the approximately $696 million

in new awards in fiscal year 1998. We also compared agency awards data

with peer review scores and the legislative objectives of each program.

App. I provides a more detailed discussion of our scope and methodology

and information about the nine programs; app. II provides an analysis of

grantee characteristics for specific programs. We conducted our work

between May 1999 and February 2000 in accordance with generally

accepted government auditing standards.

Results in Brief Education’s redesigned grant awards process has helped the agency

improve the timeliness of awards and provide better service to applicants

and grantees. Prior to the redesign, the process included several repetitive

steps and review layers, which impaired the agency’s ability to award funds

to grantees in a timely manner. As a result of the redesign, Education

provides a greater percentage of new awards earlier in the year, which

allows grantees more time to prepare for implementing their projects. For

example, in fiscal year 1999, Education awarded 57 percent of its new

awards by the end of May, compared to 12 percent for the same time period

2

See Peer Review: Reforms Needed to Ensure Fairness in Federal Agency Grant Selection

(GAO/PEMD-94-1, June 24,1994). Potential weaknesses include, among other things, the fact

that reviewers lack relevant expertise and have a conflict of interest with the applications

they review. The framework and its adaptation for this report are further discussed in app. I.
B-282710

Page 5 GAO/HEHS-00-55 Education Discretionary Grants

prior to the redesign. Also, Education has made grant information more

available and accessible to applicants and increased grantees’ flexibility to

make certain administrative changes to their projects without prior

approval. Education is considering making additional changes to the

process, which could increase efficiency and reduce costs. However, these

changes could present tradeoffs. For example, one proposal that would

allow programs to hold separate competitions for inexperienced applicants

could be viewed by other applicants as unfair because it could reduce the

amount of funding that would otherwise be available for the general

competition.

Education’s peer review process, which in fiscal year 1998 cost

approximately $2.4 million (or about 41 percent of the funds authorized for

peer review) for the nine programs we reviewed, has many of the

management controls necessary to help ensure that reviewers fairly assess

the merit of applications. For example, Education seeks to select peer

reviewers who have relevant experience, requires them to certify they are

free of actual or perceived financial conflicts of interest, and trains them to

apply evaluation criteria to score the merit of grant applications. However,

weaknesses in some management controls could raise concerns about the

fairness or perceived fairness of the process. For example, Education

generally does not require reviewers to certify that they have no

nonfinancial conflicts of interest (that is, personal and professional

conflicts) and it lacks guidance to assist officials in dismissing poorly

performing peer reviewers. Moreover, few of the programs assessed the

effectiveness of management controls to ensure that the process was

conducted fairly. For example, although several programs collected

information about the peer review process, few analyzed peer reviewers’

feedback on their review experiences or tracked the performance of

reviewers. Also, none of the programs assessed the effect that some peer

reviewers’ scoring variations had on applicants’ chances to receive funding.

We make recommendations to Education to enhance and better assess the

effectiveness of its controls.

Grant awards for the programs we reviewed were generally consistent with

peer review scoring, with funds typically awarded to applicants with the

highest peer review scores. Awards also were consistent with legislative

objectives for each program and specific measurable objectives, where

specified. For example, the objective of two programs was to improve

schoolwide programs for limited English-proficient students at schools

serving these students. About half the grantees in one of these programs
B-282710

Page 6 GAO/HEHS-00-55 Education Discretionary Grants

and about one-third in the other had 5 to 20 times the national average

number of students with limited English proficiency.

Background The majority of Education’s $38.4 billion fiscal year 1999 total budget

funded formula grants, which award funds noncompetitively to applicants

who meet certain program requirements. For example, one such program

provides grants to states to fund local adult education programs and

literacy services. Education also distributed approximately $1.7 billion in

competitive financial assistance, in the form of discretionary grants to

eligible grantees, which included local education agencies, colleges and

universities, and other applicants.3 Education’s six principal offices are

responsible for administering these discretionary grant programs. These

offices are the Office of Bilingual Education and Minority Languages

Affairs, the Office of Educational Research and Improvement, the Office of

Elementary and Secondary Education, the Office of Postsecondary

Education, the Office of Special Education and Rehabilitative Services, and

the Office of Vocational and Adult Education. Table 1 lists the nine

programs we reviewed, by principal office. Within each principal office,

there are multiple program offices which administer the agency’s

discretionary grant programs on a daily basis. The program office is

responsible for the entire grant awards process, which consists of

publishing the grant awards announcement, coordinating peer reviews to

assess the merit of applications, making funding recommendations, and

awarding grants to successful applicants. To fund these peer review

activities, Education generally can use up to 1 percent of program

appropriations.

3

Discretionary grants also include cooperative agreements which Education awards when it

anticipates having substantial involvement in working with a grantee to achieve project

outcomes.
B-282710

Page 7 GAO/HEHS-00-55 Education Discretionary Grants

Table 1: Discretionary Programs Reviewed

Peer review, the foundation of Education’s discretionary awards process,

seeks to identify high-quality projects most worthy of funding. Each grant

program has a separate peer review competition in which individuals,

largely from outside the government, review and score applications,

usually at a central review location. The number of individuals selected to

serve as peer reviewers depends on the number of applications received for

a specific grant awards competition. These peer reviewers are usually

grouped together in panels of three or more members to review

applications. Therefore, the number of peer reviewers selected determines

the number of panels that will be used for each program competition. Each

peer reviewer independently reads and scores a group of applications

randomly assigned to the panel, generally using a numerical scoring

system, against program criteria based on legislative and regulatory

requirements. After this, the reviewers convene in their respective panels

to discuss each reviewer’s scores and the strengths and weaknesses of each

application. Reviewers may modify their original scores based on the panel

discussion, but are not required to agree on a common score.

During the peer review process, program officials review all material for

accuracy and completeness and, when the review is completed, conduct a

final review of all materials. Program officials develop a single score for

each application—usually by averaging the scores of all peer reviewers on

Principal office Programs

Office of Bilingual Education and Minority

Languages Affairs

• Bilingual Education Comprehensive School

Grants

• Program Development Implementation

Grants

Office of Educational Research and

Improvement

• 21st Century Community Learning Centers

• Fund for the Improvement of Education:

State Partnerships for Character Education

• Technology Innovation Challenge Grant

Program

Office of Elementary and Secondary

Education

• Safe and Drug Free Schools and

Communities: Model Demonstration

Programs to Create Safe and Orderly

Learning Environments in Schools

• Magnet Schools Assistance

Office of Postsecondary Education • Strengthening Institutions Program

• Talent Search Program
B-282710

Page 8 GAO/HEHS-00-55 Education Discretionary Grants

the panel that reviewed the application or, less frequently, using a statistical

technique to equalize unusual scoring variances among reviewers. Program

officials may add points to the score if an applicant meets certain program

priorities, such as being in an empowerment zone or enterprise community.

The program office uses the final score to develop a rank-ordered listing of

applications reviewed by all panels, from the highest-scoring to the lowestscoring application. Once the rank order list is completed, each office

develops a cutoff point on the list, based on the appropriations for the

program. Next, each program office performs a cost analysis on those

applications that scored high enough to be considered for funding to

ensure that the costs of an applicant’s budget are justified. After the cost

analysis is completed, the program office creates a list of applications that

it recommends for funding and includes recommended funding levels. This

list is forwarded for approval, usually to the senior management level. Once

it is decided who will receive funding, awards are made.

Education Has

Improved the Grant

Awards Process

Education has achieved many of the goals of its redesigned discretionary

grant awards process, which were to increase the timeliness of awards,

streamline the process, and increase access to and availability of grant

information, among other improvements. In addition, Education has

proposed to make further changes to the process that could offer several

advantages, but could also present tradeoffs. For example, one proposal

that would allow programs to hold separate competitions just for

inexperienced applicants could reduce the amount of funding for the

general competition.

Redesign and Other Efforts

Improve Awards Process

Before October 1996, Education administered its discretionary grant

awards process through a 487-step process that took about 6 months from

the time Education publicized the awards to the time grantees received the

funding. The process involved numerous duplicative administrative steps

and review layers, which left Education staff little time to address applicant

and grantee concerns. In December 1995, Education began to redesign the

process to improve the timeliness of awards and improve Education staffs’,

applicants’, and grantees’ satisfaction with the process. Some of the major

changes resulting from the redesign include eliminating unnecessary and

time-consuming processes, decentralizing the grants-making function and

providing a single point of contact for grants administration and

programmatic information, and establishing partnerships with grantees to

help ensure successful project outcomes. In order to accomplish these

goals, Education reported in 1995 that it would reduce the 487-step process
B-282710

Page 9 GAO/HEHS-00-55 Education Discretionary Grants

to 221 steps. During our review, however, Education officials reported that,

because they had moved to a more results-oriented approach, the focus on

individual steps was no longer relevant. Ultimately, however, Education

reported that it has to date reduced the number of steps to 192. Education

officials stated that many of the steps in the redesigned process are

different from those in the original process and some are actually optional.

Table 2 summarizes the six specific goals of the redesign, the actions taken,

and the resulting changes.

Table 2: Goals of Education’s Redesign Process and Actions Taken

a

The OMB clearance process has been reduced to 60 days for competitions in which programs use

their own program-specific criteria to run the competition for discretionary grants. The process for

programs that use existing Education criteria has been reduced to 10 days.

Redesign goal Action taken and results

Reduce unnecessary, burdensome, and time-consuming

administrative steps and approval layers, which included a

commitment to reduce the 487-step process to 221 steps

• Reduced the number of steps in the awards process to the current total

of 192,

• Reduced the time needed for Office of Management and Budget (OMB)

clearance and approval of application packages used by applicants

when submitting grant proposals from 120 days to 60 days or 10 days,

depending on the criteria used for the discretionary grant competition,a

and

• Trimmed about 30 days from the approval process for Technical Review

Plans, which describe the procedures used to review applications.

Reduce the time needed to make new discretionary grant

awards, including a specific target of making 48 percent of

new fiscal year 1999 awards by the end of May

• Awarded 57 percent of the new fiscal year 1999 awards by the end of

May, compared to 12 percent for the same time period before the

redesign; earlier awards provide grantees with more time to implement

their projects

Establish a single point of contact (“one-stop shopping”) for

both programmatic and grants management information to

better serve potential applicants and grantees

• Education decentralized the grants-making function from its Grants

Division to its six principal offices.

• Grant teams in each office serve as a single point of contact for

applicants and grantees.

Increase the availability of and access to grant information • Through a website, grantees can have immediate access to grant

information, and applicants can identify funding opportunities and

retrieve grant applications.

• Education automated the payment process for discretionary grants

through its Grants Administration and Payment System (GAPS) to

provide grantees with timely access to information about their grants

and payments and to help program offices to track grantee expenditure

activity.

Provide grantees with more flexibility to manage their projects • Through its expanded authorities for discretionary grants, Education

allows grantees a one-time project extension of up to 1 year, without

prior approval, among other flexibilities.

Establish partnerships with grantees to facilitate successful

project outcomes

• Education now holds postaward conferences with some grantees on

complex or large projects to agree on expected project outcomes.
B-282710

Page 10 GAO/HEHS-00-55 Education Discretionary Grants

A preliminary independent survey of applicants and grantees, after these

changes were implemented, indicates that the majority of them are more

satisfied with various phases of the grant process, such as the availability

of information about grants and the application process.4 Education staff

report that the increased timeliness in making grant awards has also

allowed more time to address applicant and grantee concerns.

Further Changes to the

Process Present Tradeoffs

Education is considering proposals to amend its regulations that govern

discretionary grant programs to provide greater opportunity for

inexperienced applicants to compete for funding and increase the options

available for reviewing and selecting grants. According to Education

officials, these proposed changes offer additional options for reviewing and

selecting grants and are not mandatory for use in any Education program.

Before implementing these proposals, Education officials stated that they

will publish a Notice of Proposed Rulemaking in the Federal Register and

request public comment. Prompted by recent statutory requirements to

provide for the optional use and acceptance of electronic documents, such

as the Government Paperwork Elimination Act of 1998 (P.L. 105-227),

Education is also considering initiatives to automate the application and

review process. Implementing these changes could offer certain benefits,

but could also have drawbacks. Education is considering the following

changes:

• Giving special consideration to grant applicants who have never

received a grant or who have not had an active discretionary grant from

the federal government in the 5-year period prior to the application

deadline. Education had previously considered allowing programs to set

aside 10 percent of their funds for inexperienced applicants but

determined that such a threshold would not serve the best interests of

programs. It is now considering allowing programs to hold a separate

competition just for inexperienced applicants or giving preference to

these applicants in the general competition by adding points to their

scores. This would address inexperienced applicants’ perception that

they are at a disadvantage in competing with experienced applicants for

funding. On the other hand, allowing programs to hold separate

competitions for inexperienced applicants could reduce the amount of

4 Schulman, Ronca, and Bucuvalas, Inc., Survey of Customer Satisfaction with the

Department of Education Discretionary Grant Process (Draft) (Sept. 27, 1999).
B-282710

Page 11 GAO/HEHS-00-55 Education Discretionary Grants

funding that would otherwise be available for experienced applicants to

receive.

• Offering reviewers the option of rating the merits of applications using a

qualitative description ranging from excellent to unacceptable, rather

than a numerical score. Under this proposal, applications of comparable

merit would be grouped together in quality bands, thereby allowing

Education to select any of the applications in the band. Selecting

projects in this manner, however, may lead to a less defensible

justification for applications that were not selected because of the

increased discretion in drawing distinctions between applications.

While Education officials believe that using a qualitative rating that

relies on detailed reviewer comments would have more than adequate

justification, the greater degree of discretion could make it more

difficult to explain why some applications in the same quality band were

selected over others.

• Providing applicants with the opportunity to file applications on-line to

Education’s database via the agency’s website, which could expedite the

applications process by eliminating the need for Education to manually

log applications and send acknowledgement letters to applicants

through the mail.5 However, applicants are concerned that data could be

lost or changed during transmission and that the transmission of the

application could be delayed, causing them to miss the application

deadline.

• Providing programs with the option of conducting peer reviews on-line,

which could eliminate staff time spent on peer review logistics, such as

renting hotel space and the costs of physically convening reviewers in a

central location to review applications. Under this initiative, Education

would send applications to peer reviewers electronically, which would

allow reviewers to review applications at their homes or offices.

Education officials believe that electronic reviews would give Education

staff greater capability to monitor reviewers performance. However,

conducting reviews on-line could make the process vulnerable to

system failures and delays. Additionally, on-line reviews could change

the nature of panel deliberations that are key to this process. Further,

for competitions with large numbers of reviewers, it could be a

challenge for Education to effectively monitor the activities and

discussions of reviewers, given the large volume of information

exchanged.

5

As of March 2000, Education had five competitions under way that allowed applications to

be electronically initiated, developed, and submitted.
B-282710

Page 12 GAO/HEHS-00-55 Education Discretionary Grants

Management Controls

Help to Ensure a Fair

Peer Review Process,

but Could Be Improved

Education’s peer review process includes many of the management

controls needed to help ensure that applications are reviewed fairly. The

nine programs we reviewed conducted their peer reviews similarly and

program officials reported that they spent $2.4 million to conduct their peer

reviews in fiscal year 1998. To be fair, a peer review process must have

several management controls, such as measures to ensure that reviewers

possess relevant expertise and are unbiased in their reviews of grant

applications. In addition, government standards require measures to be in

place to assess the effectiveness of existing controls. However, weaknesses

in controls in some areas raise concerns about the actual or perceived

fairness of the process. For example, Education generally does not require

that peer reviewers certify that they are free of nonfinancial conflicts of

interest and does not have guidance on how to handle poorly performing

peer reviewers and what to do with their scores. Also, Education rarely

assesses information from the peer review process to ensure that controls

are properly applied.

Many of the Necessary

Controls Are in Place

Based on a framework for judging the adequacy of the peer review process

that we adapted from our earlier work, a fair peer review process must

have controls in several key areas, which include selecting reviewers with

relevant expertise, assessing peer reviewers’ conflicts of interest, training

reviewers to appropriately apply program criteria when evaluating

applications, and exercising appropriate oversight to ensure that

applications are properly evaluated. We found that Education has

management controls in place to help ensure fair reviews of discretionary

grant applications. In addition to key controls, government auditing

standards establish the need for management controls to measure, report,

and monitor program performance. For peer review, we believe this means

assessing existing controls to ensure that the peer review process is

conducted fairly. Further, Education’s own guidance instructs program

officials to annually conduct a review of their process for selecting and

using peer reviewers to ensure that qualified reviewers are used and that

the review of all applications is conducted efficiently, competitively, and

fairly. However, Education has not applied its controls as prescribed by the

auditing standards or by its own internal guidance.

We found that the nine programs selected outside reviewers with expertise

relevant to the grant award competition they served. In our review of

selected peer reviewer resumes, we found that reviewers had backgrounds

and training relevant to the competition. For example, we found that
B-282710

Page 13 GAO/HEHS-00-55 Education Discretionary Grants

reviewers with backgrounds in drug and social violence prevention and

youth development activities reviewed applications for a program whose

purpose is to provide safe and drug-free environments for students.

To help ensure that applications are fairly reviewed, Education requires

that reviewers certify that they have no actual or perceived financial

conflicts of interests, in accordance with Education’s Departmental

Directive, dated June 23, 1992. According to the directive, a reviewer has a

conflict of interest or the appearance of a conflict of interest if, for

example, the reviewer, his/her spouse, minor child, partner, or organization

would receive or potentially receive financial benefit from a discretionary

grant award. We reviewed the conflict-of-interest statements that reviewers

were required to sign for eight programs and found the reviewers’ had

certified that they had no financial conflicts.6 Programs use a variety of

methods to avoid potential financial conflicts of interest, such as asking

reviewers’ to identify conflicts when applications are assigned to them and

not allowing reviewers to read applications from states in which they live

or work. If conflicts of interest are discovered during the course of a

review, a program may choose not to use the reviewer, reassign the

application or applications with which a reviewer has a conflict to another

panel, or request a conflict-of-interest waiver. Conflict-of-interest waivers

may be granted on an individual or group basis, when it is determined that

reviewers’ financial interests are not so substantial that they would affect

the integrity of reviewers’ services. These waivers are approved by senior

principal office officials, with the concurrence of Education’s Office of

General Counsel. Eight reviewers reported conflicts of interest in two

programs. For the three reviewers in one program, officials reassigned the

applications to other panels. In the other program, the five reviewers

served under an approved conflict-of-interest waiver.

To further provide for fair peer reviews, Education trains staff and peer

reviewers on various aspects of the peer review process. Staff receive

training on the criteria that peer reviewers use to score applications, how

to review peer reviewers’ comments, and other topics. Peer reviewers

receive training during orientation sessions in various areas, such as how

to score applications against program criteria and how to document the

justifications for their scores.

6

One program was unable to locate reviewer conflict-of-interest statements.
B-282710

Page 14 GAO/HEHS-00-55 Education Discretionary Grants

Finally, Education has several ways to help ensure that peer reviewers

correctly apply scoring criteria. For example, program staff periodically sit

in on panel discussions to ensure that peer reviewers consider the

appropriate criteria when evaluating applications. Also, Education staff

check peer reviewers’ individual scoring sheets to ensure that their scores

have no math errors, adhere to program criteria, and sufficiently document

the strengths and weaknesses of the application. If the rationale for peer

reviewers’ scores is not sufficiently documented, some program officials

ask reviewers to provide additional written comments on the strengths and

weaknesses of the applications. According to Education, unsuccessful

applicants and grantees have both stressed the importance of receiving

thoughtful, substantive reviewer comments that identify the strengths and

weaknesses of the proposed projects. According to Education,

unsuccessful applicants have said that weaknesses cited by reviewers help

them strengthen their applications for the next competition. Education also

reported that grantees have said that receiving substantive reviewer

comments before the start of their grants provides them with useful

information to improve the implementation of the projects.

Weaknesses in Controls

Could Raise Concerns

About Fairness

We identified weaknesses in several key controls that may raise concerns

about the fairness or perceived fairness of the process. These weaknesses

include limitations in Education’s efforts to fully assess reviewers’ potential

nonfinancial conflicts of interest, absence of guidance for replacing poorly

performing peer reviewers, and insufficient efforts to assess the

effectiveness of existing management controls to ensure fairness in the

process. We found that Education’s guidance for making conflict-of-interest

determinations does not clearly require that program officials assess for

nonfinancial conflicts—for example, personal and professional

relationships—which could also impair their objectivity, and that conflictof-interest certifications reviewers are required to sign, in most cases, only

attest to the absence of reviewers’ financial conflicts. Nonfinancial

conflict’s can include such situations as a close friendship between the

reviewer and applicant. A February 1999 Education study of one principal

office’s peer review process recommended that the office’s conflict-ofinterest determinations go beyond financial interests because of findings

that some peer reviewers had prior professional relationships with

applicants whose applications they were reviewing.7 The study reported

that such relationships could predispose reviewers to judge applicants

based on those prior relationships, rather than solely on the merit of the

application. To preserve the integrity of the process, it is important that
B-282710

Page 15 GAO/HEHS-00-55 Education Discretionary Grants

Education require reviewers to fully disclose all potential conflicts that

could affect their reviews.

Additionally, Education has no guidance to assist program officials in

determining situations in which a peer reviewer and his/her scores should

be dismissed due to poor performance or for documenting such situations

and the reasons for any actions taken. The two programs we reviewed that

encountered this situation used different approaches to address the

problem. One program threw out the reviewer’s scores for one application

and allowed another panel to score it. The other program used a

replacement reviewer to read and rescore the applications, which changed

some applicants’ scores as well as who received awards. In both situations,

there was no documentation to support the actions taken. Without clear

guidance, program officials may not be able to apply a consistent approach

to deal with and document these situations, which can leave Education

vulnerable to criticisms that applicants are not treated fairly.

While many programs collected information from the peer review process,

they rarely analyzed or tracked the information to identify weaknesses in

the process. For example, seven of the nine programs we reviewed

collected peer reviewers’ feedback on their review experiences, but only

two analyzed the information to determine areas where improvement

might be needed. Analyzing peer reviewers’ feedback on their experiences

could provide programs with valuable information to identify weaknesses

in training and other areas that could potentially affect reviewers’ ability to

perform adequate reviews. For example, some reviewers for one program

that did not analyze peer reviewers’ feedback expressed concern that not

enough time was allowed to complete their reviews. In addition, five of the

nine programs documented information on peer reviewer performance, but

only two programs tracked the information to flag peer reviewers whose

performance was not satisfactory.

Finally, Education recognizes that some reviewers and panels may have a

tendency to score applications higher or lower than others. However, it has

done little to assess the extent to which this occurs and how such scoring

tendencies could affect applicants’ chances to receive funding and affect

the actual or perceived fairness of the process. In the absence of these

7

August and Associates, a subcontractor of the American Institutes for Research,

Strengthening the Standards: Recommendations for OERI Peer Review, Final Report (Feb.

3, 1999).
B-282710

Page 16 GAO/HEHS-00-55 Education Discretionary Grants

assessments, we analyzed peer review scoring to determine if there were

patterns of variation among panels. We found differences in the level of

scores received by applications that had been reviewed by different panels.

Applications reviewed by some panels consistently received scores that

would probably be considered high, while applications reviewed by other

panels consistently received much lower scores. The scores received by

applications in one of the eight programs we reviewed illustrate this. The

review process for that program involved 16 panels, with 15 panels each

reviewing 8 applications and 1 panel reviewing 7 applications; the

maximum score an application could receive was 145 points. One panel

gave each of its eight applications a score of 138 or higher. By comparison,

three other panels each scored only one of their applications that high and

one panel did not score any of its applications that high (see table 4 in app.

I). This suggests that some panels might have been more generous in their

scoring than others. Because applications are randomly assigned to

reviewers, it would be expected that if all reviewers had applied scoring

criteria in the same way, the applications reviewed by all panels would have

had similar distributions of high, medium, and low scores.

An application’s score is a major determinant of its ranking, which serves

as the basis for funding decisions. Therefore, application scores play a

major role in determining which applications will be funded. Table 4 in app.

I illustrates how the score an application receives affects its chances to

receive funding. For example, in the program discussed here (identified as

program #1 in the table), all eight of the application’s reviewed by the panel

that assigned the highest scores were at or above the funding cutoff score;

of the eight applications reviewed by each of the three panels that assigned

no more than one score of 138 or higher, only one was at or above the

funding cutoff score. We define the funding cutoff score as the score of the

lowest-scoring application selected for an award. It should be noted,

however, that a score at or above the cutoff level does not necessarily

guarantee funding. For example, one of the programs we reviewed selected

only a certain number of its highest-scoring applications from each panel

for grants.

To compensate for the tendency of some reviewers to score applications

higher or lower than others, Education encourages program officials to use

statistical standardization. Statistical standardization is a statistical

procedure used to correct the effect of differing reviewer approaches to

scoring. However, this procedure can only be used when certain criteria are

met, such as a large competition. While Education reported that this

procedure has been validated as sound by its expert statistician, program
B-282710

Page 17 GAO/HEHS-00-55 Education Discretionary Grants

officials told us they often do not have enough applications to meet the

requirements for standardization and, when they have used the procedure

in the past, it has resulted in the funding of mediocre applications over

higher-quality ones. However, Education officials said that standardization

is generally not used alone and may be used in conjunction with other

devices to further eliminate scoring variances across applications.

Costs of Peer Review

Activities

Program officials for the nine programs we reviewed reported that they

spent $2.4 million to conduct these peer review activities, or about 41

percent of the $5.8 million authorized in fiscal year 1998 for peer review.

Eight of the programs are authorized to use up to 1 percent of program

appropriations to pay the fees and expenses for outside review activities,

while one program is limited to one-half percent of program appropriations

for such expenses. The programs used their funds to pay for allowable

expenditures such as honorariums ranging from $100 to $125 per day,8 and

reviewers’ travel expenses. These funds were typically not set aside in

distinct accounts, but remained in the general program appropriation for

programs to draw from as needed. Some programs reported that they used

remaining funds to provide additional awards and to fund outreach efforts

to increase the visibility of programs, such as state directors’ meetings and

project directors’ meetings.

Grant Awards Are

Generally Consistent

With Peer Review

Scoring and Reflect

Legislative Objectives

Based on the programs we reviewed, Education typically made grant

awards that were consistent with the results of peer review scoring.

Moreover, the grant awards generally reflected the legislative objectives for

the nine programs we reviewed. Also, in those cases where the programs’

legislation specified that grantees have certain characteristics, we found

that the grantees generally possessed the characteristic specified.

Funding Decisions Are

Generally Consistent With

Peer Review Scoring

Results

For the programs we reviewed, Education’s funding decisions for grant

awards generally were consistent with the results of peer review scoring,

with awards predominantly given to applicants who received the highest

peer review scores. In four of the nine programs, peer review scores alone

determined who would receive funding. In these programs, officials used

8

One program paid reviewers a flat rate of $600 for the entire review.
B-282710

Page 18 GAO/HEHS-00-55 Education Discretionary Grants

the average score for each application to rank order the applications from

high to low and developed a list of those applicants recommended for

funding (including the size of the award). This meant that, in one program

with more than 300 applicants, the highest-scoring 68 applicants were

recommended to receive funding. Senior management officials usually

reviewed these lists and, in all instances, approved the recommendations.

In the remaining five programs we reviewed, applications that met certain

program priorities were awarded additional points. For example, one

program added points to the scores of applicants who had prior experience

with the program. Then applications were ranked from the highest-to

lowest-scoring and a list of applications was recommended for funding

(including size of the award) and approved, usually by senior management

officials. While the addition of priority points may have changed the rank

order of applications, peer review scores still played a key role. We found

that at least 70 percent of the award recipients in three of the four

programs would have received awards even without the additional points

based on their average peer review scores. In the fourth program,

applicants were not only awarded priority points, but program officials

were also allowed to select certain applicants over others in order to

provide geographic distribution (in accordance with program objectives) in

the awarding of grants. Even in this case, about 98 percent of the applicants

who received the highest peer review scores received funding. In the fifth

program, grant awards were made in two allocations. In the initial

allocation, 98 percent of applicants receiving awards had the highest peer

review scores. In the second allocation, the program added 10 points to the

score of applications that were not recommended for funding in the initial

allocation and that had not received a grant in the previous award cycle and

then reranked the applications. The program made awards using criteria

specified in their guidance, which resulted in 85 percent of applicants with

the highest adjusted peer review scores receiving awards. Because peer

review scores are a major determinant in awards decisions, the need for

strong and effective management controls and assessment efforts to ensure

that applications receive a fair review is evident.

Grant Awards Generally

Reflect Programs’

Legislative Objectives

The grant awards we reviewed were generally consistent with the

programs’ overall legislative objectives. We reviewed how Education

translated the legislative objectives into program requirements in (1) the

application announcement, which outlines the type of projects being

solicited and who is eligible to apply; and (2) scoring criteria, which explain

how peer reviewers should evaluate each application. We found that
B-282710

Page 19 GAO/HEHS-00-55 Education Discretionary Grants

Education adequately translated the legislative objectives into

requirements in the grant announcement and scoring criteria for each of

the nine programs we reviewed. For example, one program’s objectives

were to enhance the nation’s efforts to prevent the illegal use of drugs and

violence among, and promote safety and discipline for, students at all

educational levels. The application materials for this grant stated that

applications should “comprehensively address multiple factors that

predispose youth to drug use and violent behavior,” and the scoring criteria

emphasized the quality and significance of proposed projects. If Education

requires applicants to adhere to the guidance provided to them in the

application package, and peer reviewers apply the scoring criteria, peer

review scores will likely reflect legislative objectives. If awards are based

on these scores and priority points, they too will reflect legislative

objectives.

We found that grantees for the three programs for which the legislation

specified that recipients have certain characteristics generally possessed

the characteristic specified. The legislation governing two programs

specified that grantees develop or upgrade programs for limited English

proficient students for schools serving these students. About half the

awardees in one program and about one-third in the other had 5 to 20 times

the national average of students with limited English proficiency. The

legislation governing the other program specified that applicants were to

be a consortium composed of at least one local education agency (LEA)

with a high percentage of children living below the poverty line. We found

that nearly half the recipients had a greater percentage of children living in

poverty than the national average.9 App. II provides detailed information on

the characteristics of grantees and, to a lesser extent, about the

characteristics of applicants.

Conclusions The peer review process is a crucial component in determining which

applications are awarded grants. Peer reviewers’ scoring of grant

applications plays a major role in helping Education officials decide which

applications are most worthy of funding. While Education has many of the

management controls needed to provide for a fair peer review process,

weaknesses exist that could affect the actual or perceived fairness of the

9

Information was available only for the LEA that received the grant. Because that LEA might

not be the one with a high percentage of children living below the poverty line, we may be

underreporting the number of grantees with high percentages of children in poverty.
B-282710

Page 20 GAO/HEHS-00-55 Education Discretionary Grants

process. Finally, Education rarely analyzes peer reviewers’ feedback,

tracks peer reviewers’ performance, or assesses the implications of

variations in average scores among review panels, all of which could

improve the peer review process. Greater efforts in these areas could help

Education to identify problems and improve the peer review process.

Recommendations To improve the peer review process, we recommend that the Secretary of

Education take the following actions:

• amend Education’s directive dealing with conflict of interest and

applicable peer reviewer certifications to include a statement certifying

that peer reviewers do not have nonfinancial conflicts that could impair

their objectivity;

• develop written policy on the protocol that program officials must

follow to identify and document actions taken when a peer reviewer

must be dismissed due to poor performance; and

• amend Education’s directive concerning discretionary grant planning,

review, and award procedures to specify that peer reviewer feedback be

collected and analyzed, peer reviewers’ performance be documented

and tracked, and scoring variations among panels be assessed to

determine the effectiveness of peer review management controls.

Agency Comments Education provided comments to this report, which are reproduced in app.

III. Technical comments and clarifications were incorporated as

appropriate. In general, Education reported that it appreciated the

recognition of the improvements that it has made in the discretionary grant

process and that our report provides a good overview of those

improvements. Education agreed with all of the recommendations we

made to improve the peer review process and stated that it has already

developed new policy guidance, which will be issued soon. For example, it

has amended the peer review certification forms to include examples of

conflicts of interest that arise from personal, professional, and other

nonfinancial relationships. It also plans to require program offices to

establish specific procedures for replacing reviewers and documenting

why such actions were taken. Finally, it will issue interim guidance to

ensure that effective management controls are used in the review process,

including a process for assessing and documenting instances involving

wide differences in panels’ scoring of applications.
B-282710

Page 21 GAO/HEHS-00-55 Education Discretionary Grants

Copies of this report are being sent to the Honorable Richard W. Riley,

Secretary of Education, and other interested congressional committees. We

will also make copies available to others upon request.

If you have questions about this report, please call me on (202) 512-7215.

Other contacts and staff acknowledgments are listed in app. IV.

Sincerely yours,

Marnie S. Shaul

Associate Director, Education, Workforce,

and Income Security Issues
Page 22 GAO/HEHS-00-55 Education Discretionary Grants

Appendix I

Scope and Methodology ApApenednxidexiIs

This appendix discusses in detail our scope and methodology for

determining the progress that the Department of Education (Education)

has made in implementing its redesigned discretionary grants award

process, the management controls used to ensure fairness in the peer

review process and the costs of peer review, and the extent to which grant

awards were made according to peer review scores and legislative

objectives.

Scope In examining Education’s progress in implementing the redesigned awards

process, we focused on department-wide progress in accomplishing

specific redesign goals. We also reviewed nine discretionary grant

programs that awarded funds in fiscal year 1998, to examine management

controls for the peer review process, the cost to conduct peer review

activities, and the extent to which grant awards were consistent with peer

review scores and legislative objectives. We selected these programs to

cover a range of intended recipients, such as local education agencies

(LEAs), state education agencies, institutions of higher education, and

nonprofit organizations. We included programs that administer larger

competitions as evidenced by the number of applicants, number of

recipients, and dollar amounts awarded. And we included smaller

competitions with fewer applicants and recipients and smaller dollar

amounts awarded to see if there were differences in how the peer review

process was administered in programs of varying sizes.

The nine programs represented 15 percent of the agency’s 61 discretionary

grant programs making new awards in fiscal year 1998 and almost 43

percent of the approximate $696 million in new awards made in fiscal year

1998. In conducting our review, we did not independently verify agency

data; however, much of the data we received came from the Grants

Administration and Payment System, which Education uses to administer

its grants program. Table 3 contains information regarding the legislative

objectives of each of the nine programs, the total amount awarded, average

award size, number of applicants, and number of grantees.
Appendix I

Scope and Methodology

Page 23 GAO/HEHS-00-55 Education Discretionary Grants

Table 3: Fiscal Year 1998 Discretionary Grant Data for Selected Programs

Program

Total amount

awarded Average award

Number of

applications

Number (%)

receiving grants

Technology Innovation Challenge Grant Program

CFDA 84.303A $30,764,165 $1,538,208 328 20 (6)

Objective: Primarily to promote the use of

technology to support school reform, network and

telecommunication connections to improve

student learning, and professional development in

the integration of high-quality technology into the

school curriculum

21st Century Community Learning Centers

CFDA 84.287A $39,550,001 $399,495 1,998 99 (5)

Objective: To enable rural and inner-city public

elementary and secondary schools or a

consortium of such schools to plan, implement, or

expand projects that benefit the educational,

health, social service, cultural, and recreational

needs of their community

Magnet Schools Assistance CFDA 84.165A $96,500,000 $1,663,793 129 58 (45)

Objective: To provide grants to eligible LEAs or

consortia of LEAs for use in magnet schools that

are part of approved desegregation plans and that

are designed to bring together students from

different social, economic, racial, and ethnic

backgrounds

Bilingual Education Comprehensive School Grants

CFDA 84.290U $16,504,250 $261,972 401 63 (16)

Objective: To develop schoolwide programs for

limited English proficient (LEP) students that

reform, restructure, and upgrade all relevant

programs and operations within an individual

school that has a concentration of LEP students

Program Development Implementation Grants

CFDA 84.288S $5,790,825 $160,856 255 36 (14)

Continued
Appendix I

Scope and Methodology

Page 24 GAO/HEHS-00-55 Education Discretionary Grants

Objective: To develop and implement new

comprehensive, coherent, and successful

bilingual education or special alternative

instructional programs for LEP students, including

programs of early childhood education,

kindergarten through 12th-grade education, gifted

and talented education, and vocational and

applied technology education; to improve the

education of LEP students and their families by

implementing family education programs and

parent outreach and training activities designed to

assist parents to become active participants in the

education of their children; to improve the

instructional program by identifying, acquiring,

and updating curriculum, instructional and

educational software, and assessment

procedures; to compensate personnel, including

teacher aides who have been specifically trained,

or are being trained, to provide services; and to

provide tutorials and academic career counseling

for LEP children and youth

Fund for the Improvement of Education: State

Partnerships for Character Education

CFDA 84.215V $2,705,475 $270,547 35 10 (29)

Objective: To conduct nationally significant

programs to improve the quality of education,

assist all students to meet challenging state

content standards, and contribute to the

achievement of the National Education Goals

Safe and Drug Free Schools and Communities:

Model Demonstration Programs to Create Safe &

Orderly Learning Environments in Schools

CFDA 84.184J $4,156,452 $692,724 98 6 (6)

Objective: To enhance the nation’s efforts to

prevent the illegal use of drugs and violence

among, and promote safety and discipline for,

students at all educational levels

Talent Search Program

CFDA 84.044A $86,978,248 $265,989 763 327 (43)

Objective: To identify disadvantaged youths with

potential for postsecondary education; to

encourage them in continuing in and graduating

from secondary school and in enrolling in

programs of postsecondary education; to

publicize the availability of student financial aid;

and to increase the number of secondary and

postsecondary school dropouts who reenter an

educational program

Program

Total amount

awarded Average award

Number of

applications

Number (%)

receiving grants

Continued from Previous Page
Appendix I

Scope and Methodology

Page 25 GAO/HEHS-00-55 Education Discretionary Grants

Note: Data are for new grants only.

Source: U.S. General Services Administration, Catalog of Federal Domestic Assistance and

Department of Education-Grant Administration and Payment System, http://www.cfda.gov (cited Oct.

28, 1999).

Methodology To determine Education’s progress in implementing its redesigned

discretionary grant awards process, we interviewed department officials

and reviewed agency documentation on the redesigned awards process.

Specifically, we reviewed the agency’s progress in streamlining the process

and reducing the time to award grants, as well as the agency’s progress in

achieving other redesign goals. In addition, we reviewed and discussed

with agency officials further efforts to change the process, such as

expanding options for reviewing and selecting applications for funding.

To determine the extent to which peer review management controls were

in place to help ensure fair grant awards decisions and the costs to conduct

outside peer review activities, we interviewed department officials and

reviewed department documentation on the use of peer review in making

grant awards decisions in the nine discretionary grant programs. One

program was unable to locate copies of the conflict-of-interest statements

signed by peer reviewers. In our previous work, we found that when certain

elements were missing, the fairness of a peer review process could be

compromised.1 Based on that, we developed a framework of elements that

are needed to ensure a fair process: specifically, that reviewers (1) have

relevant expertise, (2) have no conflicts of interest, and (3) apply the

appropriate criteria when making assessments. In addition to the

framework, we used the Government Auditing Standards to assess whether

Education’s management controls included a system for measuring,

Strengthening Institutions Program

CFDA 84.031A $19,174,811 $281,983 366 68 (19)

Objective: To help eligible colleges and

universities to strengthen their management and

fiscal operations and to assist such institutions to

plan, develop, or implement activities including

endowment building that promise to strengthen

the academic quality of their institution

Program

Total amount

awarded Average award

Number of

applications

Number (%)

receiving grants

Continued from Previous Page

1

GAO/PEMD-94-1.
Appendix I

Scope and Methodology

Page 26 GAO/HEHS-00-55 Education Discretionary Grants

reporting, and monitoring the performance of the peer review process. We

also reviewed Education’s guidance for assessing the peer review process.

We specifically discussed with agency officials the process for selecting

reviewers, the training provided to peer reviewers and staff, the criteria

used to assess grant applications, and how peer reviewer scores are used in

making awards decisions.

To determine if there were patterns of variation among panels for each

grant competition we analyzed the results of peer review scoring. We began

by calculating the average application score for each panel. Then we

calculated the funding cutoff score for each program, which we defined as

the application score received by the lowest-scoring applicant chosen for

funding. Table 4 displays the detailed findings from one of the programs we

analyzed, numbers in bold representing scores of applications that scored

at or above the funding cutoff level. The funding cutoff score, in all cases,

does not specifically separate funded from nonfunded applications, as

some programs subsequently added points to the panel score because the

application met a program priority. We then grouped the panels into thirds

based on application scores, as shown in table 5. The three groups were (1)

panels with the highest scores, (2) panels with the middle scores, (3) and

panels with the lowest scores. Finally, we calculated the percentage of

applications in each of the three groups that scored at or above the funding

cutoff level. These percentages, along with similar findings from the other

programs, are shown in table 5.
Appendix I

Scope and Methodology

Page 27 GAO/HEHS-00-55 Education Discretionary Grants

Table 4: Results of Panel Score Analysis on One Program

Note: Score is in bold if it is at or above funding cutoff score.

Panel Score of applications assigned to each panel

Top-scoring

application

Secondhighest

scoring

application

Thirdhighest

scoring

application

Fourthhighest

scoring

application

Fifthhighest

scoring

application

Sixthhighest

scoring

application

Seventhhighest

scoring

application

Eighthhighest

scoring

application Average

Highest one-third scoring panels

A 145 145 144 144 141 141 139 138 142

B 145 144 142 141 139 137 135 130 139

C 145 143 143 142 139 136 101 none 136

D 145 144 144 141 141 140 136 64 132

E 143 139 137 137 135 128 118 117 132

Middle one-third scoring panels

F 143 141 135 131 128 119 111 106 127

G 145 145 145 144 144 143 120 22 126

H 144 142 139 135 129 118 100 91 125

I 143 134 133 133 129 122 106 89 124

J 142 137 134 129 118 111 110 102 123

K 145 144 141 138 138 122 82 52 120

Lowest one-third scoring panels

L 144 137 131 127 119 118 102 48 116

M 142 134 133 132 128 106 74 72 115

N 143 136 134 108 97 93 86 76 109

O 142 139 139 132 126 111 51 25 108

P 137 136 136 134 121 99 51 24 105
Appendix I

Scope and Methodology

Page 28 GAO/HEHS-00-55 Education Discretionary Grants

Table 5: Results of Panel Score Analysis Across Programs

a

We did not analyze the one program in our review that standardized peer review scores.

b

We excluded 11 panels that scored 12 or fewer applications.

c

This score does not contain the priority points that were awarded by program officials after peer

reviewers completed scoring. The score of 84 was the lowest panel score received by the applicant

with the lowest score that was awarded a grant.

d

This score does not contain the priority points that were awarded by program officials after peer

reviewers completed scoring. The score of 94.67 was the lowest panel score received by the applicant

with the lowest score who was awarded a grant.

e

Data presented are based on the results from the first round of scoring in a multitiered selection

process. Range scores represent panels with the lowest and highest percentage of applications

selected for a second round of scoring. The “Percentage of applications scored...” column represents

the portion of applications in each group that were forwarded to the second round of scoring.

f

This score does not contain the priority points that were awarded by program officials after peer

reviewers completed scoring. The score of 89.0 was the lowest panel score received by the applicant

with the lowest score who was awarded a grant.

We obtained information on the cost of conducting peer review activities

through discussions with program officials and a review of program

documentation. We discussed with program officials how they used the

money allowed for peer review and reviewed applicable legislation guiding

the uses of peer review funds to determine if funds were spent

appropriately.

To determine the extent to which grant awards were consistent with peer

review scores, we compared the scores with awards decisions. We

discussed any discrepancies with program officials. We also reviewed the

Percentage of applications scored at or above the funding cutoff

score among

Programa

Number of

panels

Range in

panels’

average

application

score

Funding

cutoff score

Highest one-third

scoring panels (%)

Middle one-third

scoring panels (%)

Lowest one-third

scoring panels (%)

1 16 105-142 136.7 74 40 20

2 7 39-62 87.7 23 12 5

3 80b 71-99 84c 92 79 59

4 59 68-98 95.3 37 14 11

5 3 78-80 84.3 40 30 33

6 44 29-89 94.67d 20 6 5

7e 56 0-67 N/A 43 33 17

8 26 29-83 89.0f 25 18 9
Appendix I

Scope and Methodology

Page 29 GAO/HEHS-00-55 Education Discretionary Grants

legislative objectives for each program in authorizing legislation and

relevant regulations. We compared the legislative objectives to information

contained in each program’s grant competition announcement proposal

and the Technical Review Plans. To provide information on the urban

status of grantees and applicants, we used GAPS data provided to us by

Education and matched it with Education Common Core of Data files to get

applicants’ metropolitan statistical area (MSA) or consolidated

metropolitan statistical area (CMSA) codes. If the matching did not

produce an MSA or CMSA code, we used the National Public School

District Locator available on the National Center for Education Statistics

website (http:www.ed.gov/ccdweb/school/index.asp) to find the missing

information. We classified applicants and grantees as urban if they were

located in an MSA or CMSA. Applicants who did not fall into either

category were classified as nonurban. Although we provide information on

the urban status of grantees and applicants, not all programs include a

legislative objective that emphasizes urban applicants.

We conducted our work between May 1999 and February 2000 in

accordance with generally accepted government auditing standards.
Page 30 GAO/HEHS-00-55 Education Discretionary Grants

Appendix II

Analysis of Grantee Characteristics for

Specific Programs ApendxiI

To provide general information about grantees, we focused on those five

programs which targeted LEAs as recipients. Doing so allowed us to

provide information about the grantees, such as per-pupil expenditure and

percentage of children in poverty, using readily available information.1 In

the other four programs, the grants were targeted primarily to state

education agencies, institutions of higher education, or nonprofit

organizations for which such information was not available.

Table 6 provides information about the grantees’ LEP population for two

programs in the Office of Bilingual Education and Minority Language

Affairs. Tables 7 and 8 provide information regarding grantees’ percentage

of children in poverty by program and urban status. Tables 9 and 10 detail

grantees’ per-pupil expenditures by program and urban status. Finally, table

11 provides information regarding the urban status of all grant applicants

for each of the five programs.2

Table 6: Number and Percentage of Grantees in Office of Bilingual Education and Minority Languages Affairs Programs, by

Percentage of LEP Students

Note: National average of LEP students is 1.1%. Information on LEP status is derived from 1990

Census data. Percentages may not equal 100 due to rounding.

Source: U.S. Department of Education, Grant Administration and Payment System, and Common Core

of Data, http://www.nces.ed.gov/ccd; and National Center for Education Statistics, Education Finance

Center website, http://www.nces.ed.gov/edfin/search (cited Oct. 25, 1999).

1

National Center for Education Statistics on the Department of Education website.

2

In all five programs, if grantees are identified as specific schools rather than LEAs, we

determined what district a school was located in and then provided information on that

district.

Percentages Shown in Parentheses

Program

Total

grantees

Less

than 2% 2-4.99% 5-9.99% 10-19.99% 20-49% 50-100%

Missing

data

Bilingual Education Comprehensive School

Grants 63 6 (10) 13 (21) 22 (35) 9 (14) 2 (3) 0 (0) 11 (17)

Program Development Implementation

Grants 36 11 (31) 6 (17) 9 (25) 2 (6) 0 (0) 0 (0) 8 (22)
Appendix II

Analysis of Grantee Characteristics for

Specific Programs

Page 31 GAO/HEHS-00-55 Education Discretionary Grants

Table 7: Number and Percentage of Grantees by Percentage of Children in Poverty

Percentages may not equal 100 due to rounding.

Note: National average of children in poverty is 16.7%. Information regarding percentage of children in

poverty is derived from 1990 Census data.

Source: U.S. Department of Education, Grant Administration and Payment System, and Common Core

of Data, http://www.nces.ed.gov/ccd; and National Center for Education Statistics, Education Finance

Center website, http://www.nces.ed.gov/edfin/search (cited Oct. 25, 1999).

Percentages Shown in Parentheses

Program

Total

grantees 0-10% 11-20% 21-30% 31-40% 41-50% 51%+

Missing

data

Overall

average

poverty

rate

Magnet Schools Assistance 58 7 (12) 10 (17) 31 (53) 8 (14) 0 (0) 1 (2) 1 (2) 23.8

Program Development

Implementation Grants 36 6 (17) 3 (8) 14 (39) 4 (11) 0 (0) 1 (3) 8 (22) 24.2

Bilingual Education Comprehensive

School Grants 63 4 (6) 16 (25) 18 (29) 10 (16) 0 (0) 4 (6) 11(17) 26.5

21st Century Community Learning

Centers 99 6 (6) 27 (27) 30 (30) 11 (11) 4 (4) 5 (5) 16 (16) 25.6

Technology Innovation Challenge

Grant Program 20 3 (15) 1 (5) 5 (25) 2 (10) 2 (10) 0 (0) 7 35) 26.8
Appendix II

Analysis of Grantee Characteristics for

Specific Programs

Page 32 GAO/HEHS-00-55 Education Discretionary Grants

Table 8: Distribution of Grantees by Percentage of Children in Poverty by Urban Status

Note: National average of children in poverty is 16.7%. We classified applicants as urban if they were

located in a metropolitan statistical area or a consolidated metropolitan statistical area. Applicants who

did not fall into either area were classified as nonurban. Information regarding percentage of children in

poverty is derived from 1990 Census data. Percentages may not equal 100 due to rounding.

a

One grantee in the Bilingual Education Comprehensive Schools program could not be identified as

either urban or nonurban.

Source: U.S. Department of Education, Grant Administration and Payment System, and Common Core

of Data, http://www.nces.ed.gov/ccd; and National Center for Education Statistics, Education Finance

Center website, http://www.nces.ed.gov/edfin/search (cited Oct. 25, 1999).

Percentages Shown in Parentheses

Program

Total

grantees 0-10% 11-20% 21-30% 31-40% 41-50% 51% +

Missing

data

Average

poverty

rate

percent

Magnet Schools Assistance

Urban 57 7 (12) 10 (18) 30 (53) 8 (14) 0 (0) 1 (2) 1 (2) 23.7

Nonurban 1 0 (0) 0 (0) 1 (100) 0 (0) 0 (0) 0 (0) 0 (0) 28.5

Program Development Implementation

Grants

Urban 28 6 (21) 1 (4) 12 (43) 2 (7) 0 (0) 1 (4) 6 (21) 23.8

Nonurban 8 0 (0) 2 (25) 2 (25) 2 (25) 0 (0) 0 (0) 2 (25) 25.8

Bilingual Education Comprehensive

School Grantsa

Urban 53 4 (8) 16 (30) 17 (32) 9 (17) 0 (0) 2 (4) 5 (9) 24.9

Nonurban 9 0 (0) 0 (0) 1 (11) 1 (11) 0 (0) 2 (22) 5 (56) 45.1

21st Century Community Learning

Centers

Urban 71 5 (7) 20 (28) 28 (39) 6 (8) 3 (4) 3 (4) 6 (8) 24.6

Nonurban 28 1 (4) 7 (25) 2 (7) 5 (18) 1 (4) 2 (7) 10 (36) 29.0

Technology Innovation Challenge Grant

Program

Urban 12 2 (17) 0 (0) 3 (25) 1 (8) 1 (8) 0 (0) 5 (42) 26.6

Nonurban 8 1 (13) 1 (13) 2 (25) 1 (13) 1 (13) 0 (0) 2 (25) 27.0
Appendix II

Analysis of Grantee Characteristics for

Specific Programs

Page 33 GAO/HEHS-00-55 Education Discretionary Grants

Table 9: Number and Percentage of Grantees by Per-Pupil Expenditure

Note: National average per−pupil expenditure is $5,529. Information is for 1995-96 school year.

Percentages may not equal 100 due to rounding.

Source: U.S. Department of Education, Grant Administration and Payment System, and Common Core

of Data, http://www/nces.ed.gov/ccd; and National Center for Education Statistics, Education Finance

Center website, http://www.nces.ed.gov/edfin/search (cited Oct. 25, 1999).

Percentages Shown in Parentheses

Program

Total

grantees $1-3,999

$4,000-

5,999

$6,000-

7,999

$8,000-

11,999 $12,000+ Missing data Average

Magnet Schools

Assistance 58 3 (5) 30 (52) 18 (31) 6 (10) 0 (0) 1 (2) 5,982

Program

Development

Implementation

Grants 36 2 (6) 15 (42) 7 (19) 5 (14) 1 (3) 6 (17) 7,022

Bilingual Education

Comprehensive

Schools Grants 63 2 (3) 37 (59) 15 (24) 2 (3) 1 (2) 6 (10) 5,609

21st Century

Community Learning

Centers 99 8 (8) 56 (57) 20 (20) 8 (8) 0 (0) 7 (7) 5,651

Technology

Innovation Challenge

Grant Program 20 3 (15) 5 (25) 4 (20) 0 (0) 1 (5) 7 (35) 5,975
Appendix II

Analysis of Grantee Characteristics for

Specific Programs

Page 34 GAO/HEHS-00-55 Education Discretionary Grants

Table 10: Distribution of Grantees by Per-Pupil Expenditure Controlling for Urban Status

Note: National average per-pupil expenditure is $5,529. Information is for 1995-96 school year.

Percentages may not equal 100 due to rounding. We classified applicants as urban if they were

located in a metropolitan statistical area or a consolidated metropolitan statistical area. Applicants who

did not fall into either area were classified as nonurban.

a

One grantee in the Bilingual Education Comprehensive School Grants program could not be identified

as either urban or nonurban.

Source: U.S. Department of Education, Grant Administration and Payment System, and Common Core

of Data, http://www.nces.ed.gov/ccd; and National Center for Education Statistics, Education Finance

Center website, http://wwwnces.ed.gov/edfin/search (cited Oct. 25, 1999).

Program

Total

grantees $1-3,999

$4,000-

5,999

$6,000-

7,999

$8,000-

11,999 $12,000 +

Missing

data Average

Magnet Schools

Assistance

Urban 57 3 (5) 29 (51) 18 (32) 6 (11) 0 (0) 1 (2) 5,997

Nonurban 1 0 (0) 1 (100) 0 (0) 0 (0) 0 (0) 0 (0) 5,099

Program Development

Implementation Grants

Urban 28 1 (4) 12 (43) 7 (25) 3 (11) 1 (4) 4 (14) 6,899

Nonurban 8 1 (13) 3 (38) 0 (0) 2 (25) 0 (0) 2 (25) 6,343

Bilingual Education

Comprehensive School

Grantsa

Urban 53 2 (4) 31 (58) 15 (28) 2 (4) 1 (2) 2 (4) 5,668

Nonurban 9 0 (0) 6 (67) 0 (0) 0 (0) 0 (0) 3 (33) 5,108

21st Century Community

Learning Centers

Urban 71 6 (8) 41 (58) 14 (20) 7 (10) 0 (0) 3 (4) 5,685

Nonurban 28 2 (7) 15 (54) 6 (21) 1 (4) 0 (0) 4 (14) 5,556

Technology Innovation

Challenge Grant

Program

Urban 12 2 (17) 1 (8) 4 (33) 0 (0) 0 (0) 5 (42) 5,879

Nonurban 8 1 (13) 4 (50) 0 (0) 0 (0) 1 (13) 2 (25) 6,088
Appendix II

Analysis of Grantee Characteristics for

Specific Programs

Page 35 GAO/HEHS-00-55 Education Discretionary Grants

Table 11: Percentage of Grant Applicants by Urban Status

Note: We classified applicants as urban if they were located in a metropolitan statistical area or

consolidated metropolitan statistical area. Applicants who did not fall into either area were classified as

nonurban.

a

One percent of unsuccessful applicants cannot be classified as urban or nonurban.

b

Seven percent of unsuccessful applicants cannot be classified as urban or nonurban.

c

Two percent of successful applicants cannot be classified as urban or nonurban.

d

Four percent of unsuccessful applicants cannot be classified as urban or nonurban.

e

Two percent of unsuccessful applicants cannot be classified as urban or nonurban.

f

Less than 1 percent of unsuccessful applicants cannot be classified as urban or nonurban.

Source: U.S. Department of Education, Grant Administration and Payment System, and Common Core

of Data; http://www.nces.ed.gov/ccd; and National Center for Education Statistics, Education Finance

Center website,http://www.nces.ed.gov/edfin/search (cited Oct. 25, 1999).

Program Urban Nonurban

Magnet Schools Assistance

Successful 98 2

Unsuccessfula 93 6

Program Development Implementation Grants

Successful 78 22

Unsuccessfulb 65 28

Bilingual Education Comprehensive School Grants

Successfulc 84 14

Unsuccessfuld 70 26

21st Century Community Learning Centers

Successful 72 28

Unsuccessfule 55 43

Technology Innovation Challenge Grant Program

Successful 60 40

Unsuccessfulf 66 33
Page 36 GAO/HEHS-00-55 Education Discretionary Grants

Appendix III

Comments From the Department of

Education ApendxiI

Now on page 10.
Appendix III

Comments From the Department of

Education

Page 37 GAO/HEHS-00-55 Education Discretionary Grants

Now on pages 10-11.

This was deleted from the

report.

Now on page 11.

Now on pages 10-11.
Appendix III

Comments From the Department of

Education

Page 38 GAO/HEHS-00-55 Education Discretionary Grants

Now on page 9.

Now on pages 16-27.
Appendix III

Comments From the Department of

Education

Page 39 GAO/HEHS-00-55 Education Discretionary Grants

Now on page 18.
Appendix III

Comments From the Department of

Education

Page 40 GAO/HEHS-00-55 Education Discretionary Grants
Appendix III

Comments From the Department of

Education

Page 41 GAO/HEHS-00-55 Education Discretionary Grants
Page 42 GAO/HEHS-00-55 Education Discretionary Grants

Appendix IV

GAO Contacts and Staff Acknowledgements ApendxiIV

GAO Contacts Lori Rectanus, Assistant Director, (202) 512-9847

Mary A. Crenshaw, Project Manager, (202) 512-7053

Staff

Acknowledgements

In addition to those name above, George Erhart, Brett Fallavollita, and

Gillian Martin, Senior Evaluators, and Jonathan Barker, Senior Attorney,

Office of General Counsel, made key contributions to this report.

(104970) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of

reports are $2 each. A check or money order should be made out to

the Superintendent of Documents. VISA and MasterCard credit

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone

phone. A recorded menu will provide information on how to obtain

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet,

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

To Report Fraud,

Waste, or Abuse in

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)


United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00