GAO

United States General Accounting Office

Report to the Chairman, Subcommittee 

on VA, HUD, and Independent Agencies, 

Committee on Appropriations, House of 

Representatives

September 2000 ENVIRONMENTAL 

RESEARCH

STAR Grants Focus on 

Agency Priorities, but 

Management 

Enhancements Are 

Possible

GAO/RCED-00-170

Page 1 GAO/RCED-00-170 STAR Grant Program

Contents

Letter 3

Appendixes Appendix I: Scope and Methodology 22

Appendix II: EPA Funding for Environmental Research 24

Appendix III: STAR Program Requests for Grant Applications

by ORD’s Priority Topics, Fiscal Years 1995 Through

2000 26

Appendix IV: Comments From the Environmental Protection

Agency 28

Tables Table 1: STAR Grants by EPA Strategic Goals, Fiscal Years 1995-99 11

Figures Figure 1: EPA’s Process for Awarding STAR Grants 9

Figure 2: Usefulness of 25 Completed Focused Grants to EPA

Program Officials 15

Figure 3: Percentage of STAR Funding, by Type of Grant, Fiscal

Years 1995 and 1998 25

Abbreviations

EPA Environmental Protection Agency

ORD Office of Research and Development

STAR Science to Achieve Results
Page 2 GAO/RCED-00-170 STAR Grant Program
Page 3 GAO/RCED-00-170 STAR Grant Program

United States General Accounting Office

Washington, D.C. 20548

Resources, Community, and

Economic Development Division

B-142370 Letr

September 11, 2000

The Honorable James T. Walsh

Chairman, Subcommittee on VA, HUD,

and Independent Agencies

Committee on Appropriations

House of Representatives

Dear Mr. Chairman:

The Environmental Protection Agency (EPA) has significantly changed the 

way it conducts scientific research to fulfill its mission of protecting human 

health and the environment. Responding to criticisms of its research 

quality and focus, EPA established the Science to Achieve Results (STAR) 

grant program in 1995, which is administered by EPA’s Office of Research 

and Development (ORD). The three objectives of the program are to (1) 

ensure that the agency involves the best non-EPA scientists in its research 

efforts, (2) provide useful research support to the agency’s program offices, 

and (3) train a cadre of environmental scientists for the future. The STAR 

program accounted for about 20 percent of the funds ORD obligated for 

research for fiscal year 1999. ORD awards STAR grants to scientists on the 

basis of 10 broad strategic goals as set out by EPA’s annual performance 

plan,1

 ORD’s six priority topics for scientific research at the agency, and the 

specific research priorities of each program office. STAR grants are 

available for (1) basic exploratory environmental research, (2) research 

focused on specific environmental topics (STAR focused grants), and (3) 

fellowships to graduate students to develop the nation’s capacity for 

addressing future environmental concerns. As of January 2000, about 800 

exploratory and focused grants had been awarded and 53 of them 

completed; in addition, about 600 fellowships had been awarded and about 

200 completed.

You asked us to review the status of the STAR grants program. Specifically, 

we are providing information on (1) whether funding amounts awarded for 

the grants align with EPA’s strategic goals, ORD’s research priorities, and 

1

 The Government Performance and Results Act of 1993 requires each federal agency to 

prepare an annual plan that includes the performance goals to be achieved. See table 1 for a 

complete listing of EPA’s strategic goals.
B-142370

Page 4 GAO/RCED-00-170 STAR Grant Program

program office priorities; (2) the extent to which the completed focused 

grants have provided research that is being used by EPA’s program offices; 

and (3) ways in which ORD could enhance its management of the program 

to help ensure that it meets its objectives. Because the STAR program is 

new and relatively few grants have been completed, it is too early to 

definitively judge the program’s long-term effectiveness. 

To address your questions, we analyzed program data; interviewed EPA 

officials knowledgeable about STAR grants; and reviewed studies by 

independent scientific groups—specifically, a March 2000 report on the 

STAR program by a joint subcommittee of EPA’s Science Advisory Board 

and ORD’s Board of Scientific Counselors and a draft report on EPA’s 

research management by the National Research Council.2

 To further 

examine the extent to which STAR grants were being used by EPA’s 

program offices, we compiled a list of the 53 focused and exploratory 

grants completed as of December 6, 1999, from the National Center for 

Environmental Research’s Web site, as identified by STAR program 

managers. We discussed the usefulness of 25 completed grants that were 

focused on specific environmental topics with potential users of the 

research in EPA, as identified by the STAR program managers. We limited 

our review to focused grants because they were more likely to be of more 

immediate use to the program offices than those grants related to 

exploratory research. Because providing useful research to the program 

offices is only one of the program’s three objectives, our observations on 

whether the research support is being used by the program offices should 

not be the only measure of judging the program’s overall effectiveness. We 

also recognize that scientific research can have long-term benefits but may 

not be immediately useful. Our detailed scope and methodology are 

discussed in appendix I.

Results in Brief STAR grant funding has generally been aligned with EPA’s, ORD’s, and the 

program offices’ broadly defined priorities. Since the STAR program began 

in 1995 through March 2000, about two-thirds of the approximately $415 

million in funding for STAR grants has addressed EPA’s multipurpose 

2 A Joint SAB/BOSC Report: Review of the Science to Achieve Results (STAR) Program, 

Joint Subcommittee of the Science Advisory Board and the Board of Scientific Counselors, 

March 2000; and Strengthening Science at the U.S. Environmental Protection Agency: 

Research Management and Peer Review Practices, National Research Council (draft 

report).
B-142370

Page 5 GAO/RCED-00-170 STAR Grant Program

strategic goal of ensuring that EPA uses sound science in addressing 

environmental hazards and improving environmental protection. Another 

21 percent has addressed the strategic goal aimed at achieving clean air, 

and the remaining grant funding has addressed six of EPA’s other strategic 

goals. STAR grant funds also generally have been aligned with one or more 

of ORD’s six research priorities, such as achieving safe drinking water, and 

with the program offices’ main priorities. Some program office research 

needs, such as reducing acid rain, were not addressed by STAR grants 

because EPA determined they had a lower research priority and because 

STAR funding was limited. 

Although STAR grants were generally aligned with the agency’s strategic 

priorities and those of ORD and the program offices, EPA’s program 

officials varied in the extent to which they believe the grants’ results are 

useful to them. Of the 25 completed focused grants we examined, 6 were 

being used and 4 were not expected to be used at all, according to program 

officials. For the remaining 15 completed grants, the officials were 

uncertain whether the grants would prove useful or not to their programs, 

but they stated they might be useful in the future—by serving as building 

blocks for research, for example. According to the officials, the grants 

varied in their usefulness to the program offices largely because the STAR 

program’s goals—meeting program office needs specifically and advancing 

environmental science generally—may be in conflict. 

ORD could enhance its management of the program to help ensure it meets 

its objectives in several areas. First, ORD has not consistently tracked the 

STAR grants to ensure that they are completed on time and that the results 

are available for use by EPA’s program offices. Second, according to several 

EPA program officials, ORD could improve its communication with the 

program offices during the grant process. ORD has not consistently 

obtained information from cognizant program officials when designing the 

requests for grant applications or reviewing grant proposals for relevancy 

during the grant selection process, nor has ORD adequately communicated 

the results of the completed grants. ORD recognizes these concerns and 

has several initiatives under way. These include establishing a Web site to 

improve communication and involving more program officials in grant 

application design and review. Because the STAR program is relatively new, 

ORD’s primary focus has been on establishing a framework and processes 

for the program and not on setting program criteria to measure its overall 

effectiveness. Recent reports by two independent scientific organizations 

agree that while the program is well structured to achieve its goals, 

communications between ORD and EPA’s program offices should be 
B-142370

Page 6 GAO/RCED-00-170 STAR Grant Program

improved. Accordingly, we are making several recommendations to EPA 

for enhancing its management of the STAR program. We believe these will 

help improve the program’s potential usefulness and effectiveness. 

We provided a draft of this report to EPA for its review and comment. EPA 

generally agreed with the report’s conclusions that the grants aligned with 

EPA’s, ORD’s, and the program offices’ broadly defined priorities and that 

funding for STAR grants falls within priority research areas. EPA also 

agreed with two of the three recommendations in this report and said it is 

taking steps to implement them. EPA did not agree with the basis for the 

third recommendation regarding the need for improved tracking and 

availability of final grant reports to program offices. Specifically, EPA 

stated that its review of grant files showed that more final reports were 

completed than indicated by our analysis. However, at the time of our 

review, a STAR program official charged with posting reports to the Web 

site represented that our list of final reports, compiled from those posted 

on EPA’s Web site and available to program officials, was complete. 

Because it now appears that a significant number of completed final 

reports had not been posted to EPA’s Web site, which was not represented 

to us at the time of our review, we believe the recommendation is still 

warranted. Finally, EPA identified several aspects of the draft report that it 

believed needed modification; for example, it stated that we negatively 

characterized the data on the usefulness of STAR grants to the program 

offices. We clarified and presented additional information to address EPA’s 

comments and made technical changes as appropriate.

Background Credible scientific research is critical to EPA’s decisions on environmental 

regulations. EPA relies on agency-funded research as well as on research 

funded by others, including government agencies, such as the National 

Science Foundation and National Institutes of Health, and private sources. 

Prior to the STAR program, EPA mainly funded research through 

cooperative agreements with outside entities, issued through its 

laboratories, to supplement the research EPA scientists performed. In 

addition, ORD funded a relatively small number of grants, managed by ORD 

headquarters staff, for longer-term exploratory research. 

During the early 1990s, EPA’s independent Science Advisory Board, the 

National Academy of Sciences, and other groups reviewed the status of 

scientific research at EPA. The groups recommended that EPA make 

changes to strengthen the quality of its scientific research, such as better 

balancing its short- and long-term research and taking steps to train the 
B-142370

Page 7 GAO/RCED-00-170 STAR Grant Program

next generation of scientists. EPA responded, in part, by establishing the 

STAR program in 1995. In doing so, EPA shifted much of its funding for 

external research away from noncompetitively awarded cooperative 

agreements administered by ORD’s laboratories and placed more emphasis 

on competitively awarded, peer-reviewed grants. EPA believed that this 

realignment would allow ORD scientists to spend less time administering 

contracts and more time conducting research, while the STAR program 

would involve top-quality scientists from outside EPA in helping to meet 

the agency’s research needs. 

The STAR program has three principal objectives. First, through the 

program, EPA attempts to make the nation’s academic research community 

an integral part of its research program, thereby ensuring that the agency 

has the highest quality science for its use. Second, EPA seeks to 

supplement and complement its internal research program, thereby 

supporting its mission by meeting the scientific and technical needs of the 

program offices. Finally, EPA attempts to support higher education in 

selected scientific fields, thereby developing a stronger scientific 

community for the future.

To achieve these objectives, the STAR program relies on three types of 

grants: exploratory grants, focused grants, and graduate fellowships. 

Exploratory grants are for fairly broad areas of environmental science, 

such as environmental chemistry and physics. Focused grants are targeted 

more specifically at particular research needs, such as developing 

standards for safe drinking water, that have been identified in ORD’s 

strategic plan. Focused grants can be awarded to environmental research 

“centers,” such as the Airborne Particulate Matter Centers. These centers 

comprise multiple scientists—usually from different disciplines—who are 

concerned with longer-term and cross-issue research. Finally, fellowships 

are given to graduate students in specified environmental disciplines, such 

as toxicology, chemistry, and economics. Fellowships for master’s and 

doctoral students are for 2 and 3 years, respectively. For fiscal year 1999, 

ORD obligated about $91 million in STAR grants for externally conducted 

research. This represented about 20 percent of ORD’s overall obligations of 

about $458 million for research. Other resources within and outside of EPA 

also provide funding for environmental research. Additional information on 

overall environmental research funding and funding levels for the three 

types of STAR grants is provided in appendix II. 

EPA solicits STAR grants through annual requests for grant applications. 

Proposals for exploratory and focused grants are reviewed for their 
B-142370

Page 8 GAO/RCED-00-170 STAR Grant Program

scientific and technical merit by panels of non-EPA scientists, which reject 

those that are not considered scientifically sound. Next, panels of EPA 

officials from ORD and other offices review and rank the proposals for 

relevance to their research needs. ORD then selects proposals to be 

awarded, taking into consideration both scientific merit and relevance to 

research needs. ORD follows a similar process for fellowship proposals. 

EPA’s process for awarding STAR grants is illustrated in figure 1.
B-142370

Page 9 GAO/RCED-00-170 STAR Grant Program

Figure 1: EPA’s Process for Awarding STAR Grants

ORD's strategic plan

(6 priority topics)

EPA's strategic plan

(10 strategic goals)

Topic-specific 

research plan 

(developed by research 

coordinating teams)

Intramural 

research needs

Extramural 

research needs

ORD 

implementation 

plans

(developed by 

laboratories)

Requests for 

Applications written

RFAs forwarded via 

Federal Register, Internet, 

and university and 

scientific organizations

External peer 

review and 

ranking 

of proposals 

(based on 

technical merit)

Internal relevancy review

(performed by agencywide panel)

Proposal selection

(based on research needs and to 

complement intramural research)

Scientific 

community
B-142370

Page 10 GAO/RCED-00-170 STAR Grant Program

STAR Grant Funding Is 

Aligned With EPA’s, 

ORD’s, and Program 

Offices’ Broad Goals

Because EPA’s, ORD’s, and the program offices’ priorities are broadly 

defined, most STAR grant funding has generally been aligned with them. 

Since the STAR program began in 1995, ORD has funded grants aligned 

with 8 of EPA’s 10 strategic goals. Most of the grant funds have been aligned 

with EPA’s strategic goal of ensuring that the agency uses sound science in 

addressing environmental issues, followed by the goal aimed at achieving 

clean air. Furthermore, all of the funding for STAR grants awarded falls 

within one or more of ORD’s six priority research topics, and grants have 

generally been awarded for the highest priority needs of each program 

office. However, some lower-priority research topics, such as reducing acid 

rain, are not being funded by STAR grants or other EPA research because 

of resource constraints.

STAR Grant Funding Falls 

Within EPA’s Strategic Goals

Since 1995, when the STAR program began, and through March 2000, ORD 

has awarded about $415 million in exploratory and focused grants. Once all 

of the awards are made for fiscal year 1999, total exploratory and focused 

grant awards will amount to about $430 million. EPA has awarded these 

grants in keeping with eight program-related strategic goals that could 

benefit from STAR grant research. Table 1 shows how the $415 million in 

grant awards corresponds to EPA’s strategic goals. The table does not 

include fellowship grant awards, which amounted to about $39 million for 

fiscal years 1995 through 1999. 
B-142370

Page 11 GAO/RCED-00-170 STAR Grant Program

Table 1: STAR Grants by EPA Strategic Goals, Fiscal Years 1995−99

Note: Individual grant amounts are associated with one strategic goal. STAR grants are not applicable 

to 2 of EPA’s 10 strategic goals, “a credible deterrent to pollution and greater compliance with the law” 

and “effective management.”

Source: GAO’s analysis of EPA’s data.

STAR grants applicable to the sound science and clean air goals 

represented about 67 and 21 percent of the funds awarded, respectively. 

Since sound science is a multipurpose goal, grants funded under it 

frequently apply to another goal as well.

STAR Grants Align With 

ORD and Program Office 

Priorities

In addition to EPA’s strategic goals, ORD has established six broad priority 

research topics: safe drinking water, high-priority air pollutants, emerging 

issues, improving ecological risk assessment, improving human health risk 

assessment, and pollution prevention and new technology. These are 

generally consistent with EPA’s strategic goals. Three of ORD’s priority 

topics—improving the assessment of risks to human health, emerging 

issues, and improving ecological risk assessment—have been the focus of 

the majority of STAR grants awarded. Appendix III shows how the research 

areas of the awarded grants are aligned with the six research topics. 

ORD takes the needs of program and regional offices into consideration 

when writing the requests for grant applications and awarding STAR 

grants. ORD’s research coordination teams include program office and 

regional representatives who identify EPA’s research needs and determine 

Dollars in millions

EPA’s strategic goal Award amounts

Clean air $87.0

Clean and safe water 16.5

Safe food 2.5

Preventing pollution and reducing risk in communities, homes, 

workplaces and ecosystems

 3.0

Better waste management, restoration of contaminated waste 

sites, and emergency response

9.7

Reduction of global and cross-border environmental risks 15.6

Expansion of Americans’ right to know about their environment 3.6

Sound science, improved understanding of environmental risk, 

and greater innovation to address environmental problems

277.1

Total $415.0
B-142370

Page 12 GAO/RCED-00-170 STAR Grant Program

the research that will be conducted internally or externally. According to 

the science advisers in the offices of Air and Radiation and Solid Waste and 

Emergency Response who have served on the teams, ORD considers 

program offices’ research priorities and ensures that the offices’ top 

research needs are covered by the STAR grants. For example, because EPA 

considers particulate matter in the air to be a costly potential health risk, 

the Office of Air and Radiation has particulate matter research as a top 

priority, and ORD has awarded STAR grants for research on particulate 

matter under its topic of high-priority air pollutants. A program official in 

the Office of Air and Radiation, who is responsible for developing the 

standards, is incorporating STAR grant results into the new standards. 

Funding for STAR grants has generally been consistent with the highest 

research priorities of the program offices. Some lower-priority research 

needs are not benefiting from STAR grants, however, because of resource 

constraints. According to program officials, the likelihood that the grants 

will address a research need depends on where that need ranks in the 

program offices’ and ORD’s priorities. For example, officials concerned 

with acid rain issues, which is a lower priority for the Office of Air and 

Radiation, told us that to date no STAR grants have included research that 

can be used to meet their needs. Similarly, no STAR grants have targeted 

research on stratospheric ozone or asthma mitigation. Officials responsible 

for these areas believe they could benefit from STAR research, but their 

research needs have a relatively low priority within the Office of Air and 

Radiation. Although ORD recently decided to fund a combustion research 

project that the Office of Solid Waste and Emergency Response will use, an 

official from that office told us that the STAR program generally has 

focused on few of its research needs because waste issues are currently a 

low priority for ORD.

According to program officials, the approximately $100 million in annual 

funding for STAR grants is not sufficient to cover all of the program offices’ 

research needs that are not otherwise met by other sources, such as ORD’s 

laboratories. To compensate, the officials stated, they seek out research 

being conducted by other organizations, some in other countries, on similar 

topics that may meet their research needs. Recently, the Science Advisory 

Board’s Research Strategies Advisory Committee completed its annual 

review of the agency’s budget request for fiscal year 2001 science and 

technology funds. The committee agreed that EPA is not adequately 

funding research on certain environmental issues, such as hazardous waste 

cleanup. ORD officials concur that budget constraints limit grant awards 

and that consequently some research cannot be funded through the 
B-142370

Page 13 GAO/RCED-00-170 STAR Grant Program

program. They told us that they consider research being done by scientists 

at EPA and other agencies, such as the National Science Foundation, when 

awarding STAR grants to ensure that program offices’ research needs get 

the best coverage possible.

Usefulness of the 

Focused Grants to 

EPA’s Program Offices 

Varies, Largely 

Because of Competing 

Program Objectives 

and the Nature of 

Grants 

Although STAR grants are generally aligned with EPA’s strategic priorities 

and those of ORD and the program offices, program officials believed that 

the usefulness of the completed focused grants to their programs varied. 

They stated that they were using 6 of the grants, did not expect to use 4 of 

them, and were uncertain about the extent to which the remaining 15 

grants might be useful in the future. According to the officials, the grants 

varied in their usefulness to the programs largely because the STAR 

program’s goals—advancing environmental science generally and meeting 

program needs specifically—may be in conflict.

Usefulness of the 

Completed Focused Grants 

to EPA’s Program Offices 

Varies 

For the 25 completed grants we examined, 6 were being used by EPA 

program officials because they met a research need. For example, one 

STAR grant created an animal model emulating the development of autism 

in humans. The goal of the research was to assess how a woman’s exposure 

to certain chemicals during pregnancy could affect the fetus. According to 

officials in ORD and the Office of Prevention, Pesticides, and Toxic 

Substances, the STAR research was the first proving a relationship between 

chemicals and the development of autism. The officials are using the 

results as they develop guidelines for assessing risks to children’s health.

In contrast, program officials did not expect to use the research results 

from four grants at all. For example, a grant researching the development 

of methods to prevent wastes by auto suppliers did not provide useful 

results, according to officials in the Office of Prevention, Pesticides, and 

Toxic Substances. The officials stated that they had begun similar research 

2 to 3 years before the grant was issued and that they had researched the 

issue more extensively than the grantees did. They stated, however, that 

the grantees benefited by increasing their technical expertise in conducting 

this type of research.

Program officials believed that the extent to which the remaining 15 

completed focused grants might prove useful in the future varied. For 
B-142370

Page 14 GAO/RCED-00-170 STAR Grant Program

example, six grants, totaling over $750,000, that researched methods for 

estimating the benefit of environmental policies were not currently useful 

but were expected to be useful to the agency in the long term. Program 

officials told us that the research topic—estimating values for lives, 

ecosystems, and other entities affected by environmental policies—is an 

important one for the agency. These officials generally agreed that the 

grants contributed to an improved understanding of the overall issue of 

environmental valuation and would probably provide some basis for future 

work on the issue. One of the program officials also stated that it was 

probably prudent to fund these grants in order to advance knowledge and 

understanding of this issue, even though the grants are not providing 

research results that can be used immediately. On the other hand, the 

future usefulness of a grant investigating wet cleaning processes—instead 

of the standard dry cleaning process—is less certain, according to officials 

in the Office of Prevention, Pesticides, and Toxic Substances. The officials 

stated that the research was well done and illustrated the environmental 

benefits of wet cleaning over dry cleaning. However, they believe that the 

garment cleaning industry is unlikely to adopt the new technology because 

cleaning companies have already made costly changes in the infrastructure 

of their plants to comply with EPA regulations. The officials plan to keep 

the grant results as a reference on the garment cleaning industry in the 

event that future regulatory changes make alternative cleaning approaches 

more viable.

Figure 2 shows how program officials perceived the usefulness of the 25 

completed focused grants.
B-142370

Page 15 GAO/RCED-00-170 STAR Grant Program

Figure 2: Usefulness of 25 Completed Focused Grants to EPA Program Officials

Note: These focused grants were completed as of December 1999.

The STAR Program’s 

Competing Goals and 

Reliance on Grants Limit the 

Usefulness of the Research 

to EPA Program Officials

The STAR program’s objectives may be contradictory in that the program is 

expected to focus on achieving long-term gains in environmental science as 

well as being responsive to the more immediate needs of the program 

offices. According to some program officials, they understood that the 

foremost goal of the STAR program was to advance environmental science 

rather than to meet the programs’ needs. As a result, STAR grants may be 

limited in the extent to which they are useful to the program offices. 

Furthermore, the STAR program’s reliance on grants to accomplish 

environmental research limits its ability to directly respond to the research 

needs of EPA’s program offices. The principal purpose of any grant is to 

assist grantees in furthering a public purpose rather than to obtain services 

directly for the government’s benefit. According to several EPA program 

officials, program offices’ needs do not always match up with grant 

recipients’ research objectives; therefore, some of the STAR grant topics 

are not directly applicable. In addition, one program official told us that 

STAR researchers are sometimes reluctant to answer questions posed by 

program officials because the terms of the grant do not require them to be 

responsive to the needs of EPA’s program offices.

6 • Useful

4 • Not useful

15

•

Usefulness uncertain

Number of grants
B-142370

Page 16 GAO/RCED-00-170 STAR Grant Program

Research for STAR grants may also not be completed in a time frame that is 

useful to the program offices. STAR grants range from 1 to 5 years. 

Grantees can request a 1-year extension, which ORD officials must 

automatically approve in accordance with OMB’s Circular A-110, according 

to ORD officials, and ORD may authorize additional extensions. Thus, even 

though it may be several years before STAR grants produce results, the 

grantees are still in compliance with the terms of their grants. Program 

officials also differed in whether grant timing fit their needs. For example, 

an official in the Office of Air and Radiation told us that projecting research 

needs that far in advance was possible, while another in the Office of Solid 

Waste and Emergency Response stated that many of his office’s research 

needs involve cleanup work at Superfund sites and require results in a 

shorter amount of time than the grants can usefully provide.

ORD Could Enhance 

Its Management of the 

STAR Program to Help 

Ensure That It Meets 

Its Objectives

ORD could enhance its management of the STAR program to help ensure 

that the results of the grants are readily useful to the EPA program offices 

and that the program meets its objectives. Areas needing further 

improvements include (1) tracking grants to ensure that they are 

completed on time and have produced the research intended; (2) improving 

communications with EPA’s program offices when designing and reviewing 

grants for relevancy and disseminating grant results; and (3) establishing 

criteria to measure program effectiveness. Reports by two independent 

scientific organizations agree that while the program is generally well 

structured for achieving its goals, communications between ORD and EPA’s 

program offices should be improved.

ORD Could Further Improve 

Its Tracking of Grant 

Results

ORD has not been adequately tracking grants and has not ensured that 

exploratory and focused grants were completed on time. As of December 

1999, 144 of the nearly 200 grantees for exploratory and focused grants had 

missed their deadlines, including extensions, for submitting final reports. 

ORD officials recently noted that they plan to do a better job of tracking 

grants. In that regard, in June 1999, ORD hired a contractor to track and 

identify interim and final reports. ORD has identified a number of 

additional final reports since December 1999. According to a STAR 

program manager, however, ORD cannot ensure that research will produce 

results in the agreed time or with the intended results. Similarly, ORD has 

not adequately tracked the fellowship grants to determine whether they 

have been effective. It had not analyzed whether any of the approximately 

200 fellowships completed as of December 1999 had resulted in college 

graduates taking jobs in environmental occupational fields. Recognizing 
B-142370

Page 17 GAO/RCED-00-170 STAR Grant Program

the need to determine whether fellowship recipients are continuing in the 

environmental sciences, ORD began in early 2000 to track the recipients of 

completed fellowships. In that effort, ORD has obtained follow-up 

information for about 60 of the 200 completed fellowships. 

ORD Could Better 

Communicate With Program 

Offices About STAR Grants

ORD could also improve its communication with the program offices 

during the grant process, according to several EPA program officials. For 

example, ORD has not consistently obtained information from cognizant 

program office officials when designing requests for grant applications or 

reviewing grant proposals for relevancy during the grant selection process. 

While a number of program officials have participated in these processes, 

others told us that they have not been invited to do so. One official, who 

had not been involved in the pre-award process, believes that, had she been 

involved, she might have influenced ORD to select a grant that could have 

met her and others’ research needs. She acknowledged that ORD has 

recently recognized that more participation from the program offices prior 

to awarding the grants could increase their usefulness. The draft report by 

the National Research Council agreed that some program officials had 

expressed concern that, in practice, they have little influence on ORD’s 

research priorities. 

Furthermore, ORD had not adequately communicated grant results to 

program officials. For 13 of the 25 grants we examined, program officials 

learned about completed research as a result of ORD outreach and 

notification. For the remaining grants, officials learned about 1 grant’s final 

results through a professional conference and were not aware of the 

completed research for the remaining 11 grants until we contacted them for 

this review. In addition, most of the program officials were not aware of 

interim research results for the grants we reviewed. One program official 

explained that, if he had been aware of the interim research results on air 

pollution in large buildings for one STAR grant, he could have notified the 

researchers of problems with their methodology and corrections could 

have been made. Instead, the usefulness of the completed research is 

questionable, and unless the methodology issue can be resolved, the 

official will be unable to use the results. 

ORD has taken steps to communicate the results of STAR program 

research, including posting STAR research report summaries on its Web 

site, providing program offices with lists of grants that might contain useful 

research, and sponsoring workshops on grant topics. While the program 

officials acknowledged that ORD makes STAR information available, the 
B-142370

Page 18 GAO/RCED-00-170 STAR Grant Program

format of this information makes it difficult for program officials to identify 

research relevant to their needs. For example, program officials stated that 

interim and summaries of final research reports are organized on ORD’s 

Web site by grant topic, a format that makes identifying specific relevant 

research difficult because they have to sift through numerous lengthy 

reports. For the ORD workshops on grant topics, not all grant topics are 

covered. For example, the indoor air research topics were not covered in 

any workshop, according to a program official. ORD managers 

acknowledge ongoing difficulties in conveying research results to the 

program offices and say they have been increasing their efforts to do so.

According to both the National Research Council’s draft report and the 

March 2000 report by the joint subcommittee of EPA’s Science Advisory 

Board and ORD’s Board of Scientific Counselors, now that the STAR 

program is starting to produce results, EPA should place greater emphasis 

on communicating those results to potential users in the agency. By 

strengthening communication between STAR program managers and the 

program offices, ORD could better ensure that STAR results are being 

rapidly and effectively transferred to the officials who can use them. The 

joint subcommittee report also recognizes that ORD has initiated several 

efforts to improve communication, such as developing reports on the 

status of research topics.

ORD Has Not Focused on 

Developing Measures to 

Help Ensure the Program 

Meets Its Goals

Because the STAR program is relatively new, ORD’s primary focus has been 

on establishing a framework and processes for the program, not on setting 

criteria for measuring its overall effectiveness. For example, ORD has not 

set goals for measuring whether the grants are meeting the research needs 

of EPA’s program offices. Such measures would allow STAR managers to 

determine if the program as a whole is achieving its expected results of 

involving the best scientists in EPA research, providing useful research for 

the program offices, and training a cadre of environmental scientists for the 

future. According to ORD officials, most of their efforts to date have 

focused on establishing a high-quality research program. They now believe 

it is time to place a greater emphasis on evaluating the program’s benefits. 

Consequently, ORD asked the joint subcommittee of EPA’s Science 

Advisory Board and ORD’s Board of Scientific Counselors for assistance in 

developing measures for determining if the STAR program is meeting its 

overall goals. 

The March 2000 joint subcommittee report focused on evaluating whether 

the STAR program is structured appropriately to achieve its goals. Overall, 
B-142370

Page 19 GAO/RCED-00-170 STAR Grant Program

the report found that the program is structured and managed to generate 

high-quality science, conducted by well-qualified scientists, on topics that 

are relevant to the environmental problems identified in EPA’s strategic 

plan. The report concluded that the STAR program is generally well 

planned, organized, and managed for achieving its goals. However, it 

recommended that ORD seek expert assistance in developing a monitoring 

and evaluation system for the program. 

Conclusions Initial results indicate that the STAR program is yielding some research that 

is useful to EPA’s program offices and that it is well-structured and 

managed to generate high-quality science. However, for the STAR program 

to better meet one of its major goals—that of supporting the research 

needs of EPA’s program offices—ORD must improve its tracking of grants 

to ensure that the research is being completed as agreed and that it is 

communicating the research results throughout EPA. Reports by two 

independent scientific groups agree that the STAR program’s research 

results should be better communicated. EPA recognizes this concern and 

has taken several actions to improve communications, but more could be 

done. 

ORD has focused the majority of its efforts to date on the near-term 

objectives of establishing the STAR program and on awarding the grants. 

However, ORD has not established criteria to help ensure that the program 

is meeting its major objectives. Now that the program has begun to 

produce results, it is appropriate for EPA to focus greater attention on 

managing on-going research, disseminating research results, and evaluating 

program results to date. 

Recommendations To enhance the effectiveness of the STAR program, we recommend that the 

Administrator of EPA direct the Assistant Administrator of ORD to take the 

following actions: 

• Track and monitor the grants to ensure that interim and final research 

results are delivered on time and are made available as soon as possible 

for use by the program offices. 

• Take the additional steps needed to better disseminate and 

communicate STAR research results to the appropriate program 

officials. This would require continuing and expanding the efforts 
B-142370

Page 20 GAO/RCED-00-170 STAR Grant Program

already under way to consult with program offices in determining the 

most effective communication methods.

• Develop program criteria to evaluate the effectiveness of each type of 

grant—exploratory grants, focused grants, and fellowships. In addition, 

the criteria should assist EPA in drawing an overall conclusion on 

whether these grants satisfy the program’s overall objectives.

Agency Comments We provided a draft of this report to EPA for its review and comment. EPA’s 

comments and our detailed responses are in appendix IV. EPA agreed with 

the report’s conclusions that the grants generally align with EPA’s, ORD’s, 

and the program offices’ broadly defined priorities and the funding for 

STAR grants falls within priority research areas. However, EPA suggested 

changes in five other areas. First, EPA stated that our report negatively 

characterized the data on the usefulness of STAR grants to the program 

offices, noting that no benchmarks exist for an appropriate level of 

usefulness. We disagree. Rather, our report simply portrays the views of 

EPA program officials, one of the intended beneficiaries of the program, on 

the grants’ usefulness. We agree that no benchmarks exist for the 

usefulness of grants, although benchmarks would be helpful to evaluate the 

program. Second, EPA stated that there is no contradiction in using grants 

to support the program offices and to further advance environmental 

research. We recognize that the STAR program can contribute to both of 

these goals. However, as the draft report stated, program officials noted 

that these goals may conflict and that the primary purpose of grants is to 

assist grantees in furthering a public purpose rather than in obtaining 

services directly for the government’s benefit. Third, EPA stated that the 

report misrepresented the relative amount of STAR grant funding to overall 

ORD funding. We changed the report to more accurately reflect the 

proportion of ORD’s funding accounted for by the STAR grant program. 

Fourth, EPA stated that it did not agree with our conclusions as to the 

number of grants that had been completed and made available to program 

offices. Specifically, EPA stated that, by counting only those final reports 

posted to its Web site, we did not include a substantial number of 

additional completed grant reports that were in its files. Therefore, EPA 

disagreed with our conclusion and recommendation that tracking of STAR 

grants should be improved. While EPA states that additional research 

reports were in its files, at the time of our review, a STAR program official 

confirmed that all final reports were posted to the Web site. If they were 

not posted, they were not readily available to the program offices. 

Therefore, we believe our conclusions and recommendation are warranted. 

Finally, EPA stated that it has been taking steps to implement two of the 
B-142370

Page 21 GAO/RCED-00-170 STAR Grant Program

report’s recommendations to better disseminate and communicate STAR 

research results and to evaluate the program. We acknowledged these 

efforts in the report. 

We are sending copies of this report to appropriate congressional 

committees; interested Members of Congress; the Honorable Carol M. 

Browner, Administrator, Environmental Protection Agency; and other 

interested parties. We will also make copies available to others on request.

Please call me at (202) 512-6111 if you have any questions about this report. 

Key contributors to this report were John Wanska, Karla Springer, John C. 

Johnson, and Roger Bothun. 

Sincerely yours,

Peter F. Guerrero

Director, Environmental 

Protection Issues
Page 22 GAO/RCED-00-170 STAR Grant Program

Appendix I

Scope and Methodology ApApenednxidexiIs

To obtain information on the Environmental Protection Agency’s (EPA) 

Science To Achieve Results (STAR) grant program, we interviewed officials 

in, and collected and analyzed information from, the Office of Research and 

Development’s (ORD) National Center for Environmental Research and the 

Grants Administration Division; the Office of Air and Radiation; the Office 

of Solid Waste and Emergency Response; and the Science Advisory Board’s 

Research Strategies Advisory Committee. We also reviewed two recent 

reports by independent scientific groups. The March 2000 report by a joint 

subcommittee of EPA’s Science Advisory Board and ORD’s Board of 

Scientific Counselors objectives were to examine whether (1) the STAR 

program is structured appropriately to achieve its stated purpose, (2) the 

program is integrated effectively with the agency’s strategic plans and 

programs, and (3) efforts to communicate with the external scientific and 

regulatory communities regarding STAR research opportunities and 

outputs were adequate. The draft report by the National Research Council 

assessed the overall structure and management of EPA’s research program 

and evaluated scientific peer review procedures used by the agency.

To determine whether the grants awarded met EPA’s strategic goals and 

ORD’s priorities, we interviewed ORD officials to obtain an understanding 

of how grants are categorized by subject area and how these subjects were 

aligned with EPA’s strategic goals and priorities. We also obtained 

information on the grant award process and reviewed documentation on 

the process. The documents reviewed included EPA’s and ORD’s strategic 

plans and various listings of the grants awarded. The listings identified the 

year of the award, the research topic, and the dollars awarded. Using this 

information, we compared the grants awarded to EPA’s goals and ORD’s 

research priorities to determine how the grants were aligned with specific 

goals and priorities.

To assess the extent to which the completed focused grants have provided 

research that is being used by EPA’s program offices, we obtained 

information on those grants completed as of December 6, 1999, from the 

National Center for Environmental Research’s Web site. At the suggestion 

of STAR program managers, we considered a grant to be completed if its 

final report was posted on the Web site and the December 6, 1999, date 

provided STAR program staff time to ensure that all available final research 

reports had been posted on the site. We excluded completed exploratory 

grants, since those grants were less likely, according to STAR managers, 

than focused grants to yield results that the programs could use in the nearterm. We also obtained from a STAR manager a list of staff in the program 

offices and regions who were most likely to benefit from the results of the 
Appendix I

Scope and Methodology

Page 23 GAO/RCED-00-170 STAR Grant Program

completed grants. We contacted those officials, and selected others who 

we learned were also potential users of the results, to discuss their views 

on the research from the completed grants. Officials we interviewed were 

from ORD; the offices of Prevention, Pesticides, and Toxic Substances, 

Water, Air and Radiation, Solid Waste and Emergency Response, 

International Activities, and Policy and Reinvention; and regions IX and X.

In addition, we used data from EPA’s automated Grants Information 

Control System to compile lists of grants that should have been completed 

as of December 6, 1999. Exploratory and focused grant recipients are 

required by their grants to provide a final report to ORD within 90 days of 

the expiration of the grant’s project period. Therefore, to determine which 

exploratory and focused grants should have been completed and had final 

reports submitted to STAR managers, we considered only those with 

project periods ending at least 90 days prior to December 6, 1999. As a 

condition of the grant, fellowship recipients must submit their theses or 

other publications to ORD after the end of the grant period, as soon as 

those papers are available. Because that deadline is less definitive, to 

determine which fellowships should have been completed, we used 

December 31, 1999, as the cutoff date and asked STAR managers about any 

actions they were taking to ensure that the fellowships were effective. We 

did not independently verify the accuracy of the grants data. However, we 

discussed the issue of data accuracy with an official in the Grants 

Administration Division, who told us that the data fields we used were 

likely to be accurate because they are relatively easy for the grant project 

managers to keep updated. 

We conducted our work between September 1999 and August 2000 in 

accordance with generally accepted government auditing standards.
Page 24 GAO/RCED-00-170 STAR Grant Program

Appendix II

EPA Funding for Environmental Research ApendxiI

For fiscal year 1999, EPA obligated about $458 million for ORD, whose 

primary purpose is to provide research for the agency. About $218 million 

of this amount was for internal research ORD conducted, about $91 million 

was for the external STAR program, and about $149 million was for other 

external research, such as contracts. In addition to the ORD funding, EPA’s 

program offices fund research projects. In fiscal year 1999, about $118 

million was available for these projects, but the agency does not know how 

much of that was spent on research-related efforts. ORD is currently trying 

to identify all the research that EPA is conducting; however, ORD does not 

plan to identify the amount of funding for research conducted outside of 

ORD because of difficulties in defining and accounting for research. 

Estimates of the total federal funding for environmental research range 

from about $2 billion to $3.2 billion for fiscal year 2000.1

The relative funding levels for the three types of STAR grants—exploratory, 

focused, and fellowships—has changed from fiscal year 1995 to fiscal year 

1998. In fiscal year 1995, the year the STAR program began, EPA awarded 

approximately $58 million, with focused grants accounting for the largest 

share of funding, about $29 million, or 50 percent, of the total. In fiscal year 

1998, ORD awarded approximately $148 million in STAR grants, and 

focused grants constituted 83 percent of the total amount as shown in 

figure 3.2

1

 In April 2000, the American Association for the Advancement of Science estimated about 

$2 billion in total federal environmental research for fiscal year 1999; in 1999, the National 

Science Foundation estimated about $3.2 billion.

2

 Fiscal year 1998 was the last year for which all of the STAR grants were awarded. ORD is 

still in the process of awarding grants for requests for proposals from fiscal year 1999 and 

has not yet awarded any grants for the fiscal year 2000 applications. 
Appendix II

EPA Funding for Environmental Research

Page 25 GAO/RCED-00-170 STAR Grant Program

Figure 3: Percentage of STAR Funding, by Type of Grant, Fiscal Years 1995 and 1998

Source: GAO’s analysis of EPA’s data.

Over half of the focused grants awarded for fiscal year 1998 were for 

research centers at universities, a practice that ORD has recently begun to 

emphasize. There were no center grants for fiscal year 1995. According to a 

STAR program manager, many of the research issues that the agency faces 

currently, and expects to face in the future, require multidisciplinary work, 

and grants to centers are more suited to this type of research. Fellowship 

grants accounted for about $10 million in fiscal year 1998 and supported 

approximately 300 fellowships. ORD expects to continue funding 

fellowships at this level. Awards for exploratory grants have decreased 

significantly. According a to STAR program manager, exploratory grants are 

proving to be the least useful of the STAR grants. While the manager 

believed some of the broad research done with exploratory grants is 

beneficial to the agency, ORD has gradually diminished the exploratory 

share in favor of more focused grants.

9%

Fellowships

41% •

•

Exploratory

50%

•

Focused

7%

Fellowships

10%

•

•

Exploratory

Center Noncenter

Noncenter

83%

• •

Focused

Fiscal year 1995 Fiscal year 1998

37%

46%
Page 26 GAO/RCED-00-170 STAR Grant Program

Appendix III

STAR Program Requests for Grant 

Applications by ORD’s Priority Topics, Fiscal 

Years 1995 Through 2000 ApendxiI

ORD’s priority topics

STAR Requests for Application by 

subject matter

Safe

drinking

water

Highpriority air

pollutants

Emerging

issues

Research to

improve

ecological

risk

assessment

Research to

improve

human health

risk

assessment

Pollution

prevention

and new

technology

Air

Indoor air quality X X

Health effects of particulate matter XX X X

Air pollution chemistry and physics XX X X

Air toxics XX X

Mercury fate and transport XXX X

Water

Drinking water XX X

Risk-based decisions for contaminated 

sediments XXX

Water and watersheds X X

Health effects of arsenic X X

Ecology

Ecological assessment and indicators X X

Global climate change X X

Regional scale assessment and analysis X X

Ecology and oceanography of harmful algal 

blooms XXX

Ecosystem restoration X

Health

Exposure of children to pesticides X X

Endocrine disrupters XXX

Children’s environmental health and 

disease prevention research centers X X

Human health risk assessment XXX

Role of inter-individual variability in human 

susceptibility XXX

Children’s vulnerability to toxic substances 

in environment XXX X X

Exposure to waste combustion products XX X X

Chemical mixtures in environmental health XXX X X

Other

Analytical and monitoring methods XX X X X
Appendix III

STAR Program Requests for Grant 

Applications by ORD’s Priority Topics, Fiscal 

Years 1995 Through 2000

Page 27 GAO/RCED-00-170 STAR Grant Program

Source: EPA’s classification of grants awarded by research priorities.

ORD’s priority topics

STAR Requests for Application by 

subject matter

Safe

drinking

water

Highpriority air

pollutants

Emerging

issues

Research to

improve

ecological

risk

assessment

Research to

improve

human health

risk

assessment

Pollution

prevention

and new

technology

Environmental fate and treatment of toxics 

and hazardous wastes X XX

Environmental statistics XX X X

High-performance computing XX X X

Technology for a sustainable environment XX X

Decision-making and valuation for 

environmental policy X X

General solicitation − exploratory research XXX X X X

Socioeconomic projects related to pollution 

prevention X X

Program on bio-remediation XXX

Futures: detecting the early signals X

Total 10 12 22 18 24 11

(Continued From Previous Page)
Page 28 GAO/RCED-00-170 STAR Grant Program

Appendix IV

Comments From the Environmental 

Protection Agency ApendxiIV

Note: GAO’s comments 

supplementing those in the 

report text appear at the end 

of this appendix.

See comment 1.
Appendix IV

Comments From the Environmental 

Protection Agency

Page 29 GAO/RCED-00-170 STAR Grant Program

See comment 2.
Appendix IV

Comments From the Environmental 

Protection Agency

Page 30 GAO/RCED-00-170 STAR Grant Program

See comment 5.

See comment 3.

See comment 4.
Appendix IV

Comments From the Environmental 

Protection Agency

Page 31 GAO/RCED-00-170 STAR Grant Program
Appendix IV

Comments From the Environmental 

Protection Agency

Page 32 GAO/RCED-00-170 STAR Grant Program

GAO’s Comments. 1. We do not agree that our report negatively characterizes the number of 

grants useful to program officials. Our report states the number of 

completed focused grants that are useful, not useful, or of uncertain 

usefulness based on our discussions with EPA program officials. We 

recognize that there is no benchmark for the number of grants that should 

be useful to program officials, although one would be helpful to determine 

the effectiveness of the STAR program. Furthermore, while we agree that 

STAR research may be useful to parties outside of EPA, we did not go 

beyond interviewing EPA program and regional officials about the 

usefulness of STAR focused grants because it is a stated goal of these 

grants to address the research needs of these officials.

2. We disagree that there may not be a contradiction in using a grants 

program to simultaneously support the agency’s mission and to advance 

the understanding of complex environmental issues. We recognize that EPA 

has made efforts to direct STAR grants into research areas that are relevant 

to the program offices, and our review has shown that, consequently, some 

research results have proven useful in meeting program needs. However, 

the principal purpose of any grant is to assist grantees in furthering a public 

purpose rather than obtaining services directly for the government’s 

benefit. Furthermore, program officials stated that the needs of the 

program offices have not always matched the grant recipients’ research 

objectives and that the grant research has often been designed to 

contribute to broad scientific issues. Therefore, as the program officials we 

interviewed pointed out, the goals of the program may conflict.

3. We agree that the amounts comparing STAR program obligations with 

total ORD research obligations for fiscal year 1999 should be revised. We 

revised the report to reflect that ORD obligated $91 million for the STAR 

program out of a total of $458 million it obligated for all research. Thus, the 

STAR program accounts for about 20 percent of all research obligated by 

ORD. We do not agree that the STAR program’s obligated amounts should 

be compared to ORD’s overall budget because that amount may include 

funding for activities other than research. 

4. We disagree with EPA’s assertion that our conclusion and 

recommendation regarding the need to improve the tracking process is 

incorrect and should be revised accordingly. While EPA states that 

additional research reports were available in the files at the time of our 

review, these were not posted to the Web site and were therefore not 

readily available to the program offices. We worked with STAR program 
Appendix IV

Comments From the Environmental 

Protection Agency

Page 33 GAO/RCED-00-170 STAR Grant Program

managers to develop the methodology for counting completed grants as 

those with final reports posted to the Web site, as discussed in appendix I. 

Furthermore, we provided the officials with additional time to identify all 

completed reports in their files and to post them to the Web site. As a 

result, we believe that no changes are needed to our report. 

5. Our report acknowledges the efforts EPA has begun to develop criteria 

for evaluating the STAR program’s effectiveness and to better disseminate 

and communicate research results. 

(160505) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of 

reports are $2 each. A check or money order should be made out to 

the Superintendent of Documents. VISA and MasterCard credit 

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are 

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and 

testimony. To receive facsimile copies of the daily list or any list 

from the past 30 days, please call (202) 512-6000 using a touchtone 

phone. A recorded menu will provide information on how to obtain 

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet, 

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at: 

http://www.gao.gov

To Report Fraud, 

Waste, or Abuse in 

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

(160505) Letr

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00