United States General Accounting Office General Government Division

Washington, D.C. 20548

Page 1 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

B-286123

September 25, 2000

The Honorable F. James Sensenbrenner, Jr.

Chairman, Committee on Science

House of Representatives

Subject: Observations on the Fiscal Year 1999 Annual Program Performance Report and

Fiscal Years 2000 and 2001 Annual Performance Plans for Selected Science Agencies

Within the Department of Commerce

Dear Mr. Chairman:

In response to your April 11, 2000, request and based upon subsequent discussions we had

with your office, this letter provides our analysis of the Department of Commerce’s Fiscal

Year 1999 Annual Program Performance Report (report) and the fiscal year 2000 and fiscal

year 2001 plans (plan) for selected science-related agencies. These documents were required

by the Government Performance and Results Act (GPRA). The performance plans are

required to identify annual goals and measurable target levels for performance. The

performance report is required to show how actual performance in a particular fiscal year

compared with performance as described in that year’s performance plan. Together with the

GPRA-required strategic plan, these documents provide the framework for agencies to

manage for results. Commerce is currently revising its strategic plan, which is to cover fiscal

years 2000 to 2005. According to Commerce, the upcoming fiscal year 2002 annual

performance plan is expected to fully reflect the revised strategic goals and measures.

In conducting our analysis, we focused on the sections of the performance report and

performance plans that dealt with Commerce’s two major science agencies, the National

Oceanic and Atmospheric Administration (NOAA) and the Technology Administration (TA).

Together, the budget requests for these two agencies accounted for approximately 67 percent

of Commerce’s overall fiscal year 2001 budget request of $5.4 billion. The objectives of our

work were to determine (1) whether the NOAA and TA sections of Commerce’s fiscal year

1999 performance report and fiscal years 2000 and 2001 performance plans told a coherent

story in addressing science issues, (2) whether NOAA’s and TA’s performance goals,

budgetary and full time equivalent (FTE) staffing resource requirements identified in

Commerce’s fiscal year 2001 performance plan linked, i.e., matched, to those in NOAA’s and
B-286123

Page 2 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

TA’s sections of Commerce’s fiscal year 2001 congressional budget submission,1and (3)

whether the NOAA and TA sections of Commerce’s fiscal year 2001 performance plan

addressed the weaknesses that we identified in our review of Commerce’s fiscal year 2000

performance plan.2

Detailed information about the objectives, scope and methodology of our

work, as well as comments provided by Commerce, are discussed near the end of this letter.

Results in Brief

The NOAA and TA sections of Commerce’s performance report and performance plans

address science issues in a generally coherent and consistent manner. For example, NOAA’s

and TA’s mission statements and performance goals remained essentially unchanged in each

of the three GPRA documents. Further, NOAA and TA described their performance goal

strategies or objectives, supporting activities, and crosscutting activities in the fiscal year

2000 and fiscal year 2001 plans, and these descriptions also remained essentially unchanged.

One difference between the two plans, however, was that an additional goal of protecting the

nation’s information infrastructure was added to TA’s section of the fiscal year 2001 plan.

Also, both NOAA and TA expanded their discussions from the previous year’s plan on

departmentwide or bureau-specific initiatives. We did note, however, that significant changes

were made to the performance measures and resource requirements and that these changes

were not always explained in the GPRA documents. Although GPRA guidance does not

require that such changes be explained, without explanations the rationale for the changes

that were made and the impact that they would have on addressing science issues was not

clear. Also, in one case, no performance measures were established for a performance goal.

Additionally, performance measures were not always established for NOAA’s or TA’s

contribution to departmentwide initiatives, nor were measures established for their

respective bureau-specific initiatives. As a result, it was not always easy to determine the

progress NOAA and TA had made towards achieving their goals or in contributing to

departmentwide efforts.

Our review of the NOAA and TA sections of the fiscal year 2001 plan and their sections of the

fiscal year 2001 budget submission showed that performance goals, budgetary resources and

FTE levels were the same in those two documents in most cases but differed to some degree

in other cases. For example, we found direct linkages between the performance plan and

budget submission for NOAA’s performance goals and their supporting objectives or

activities. For TA, we found direct linkages for six of its seven performance goals, but not for

each goal’s supporting objectives or activities. And, for both NOAA and TA, there were no

linkages for any of their respective bureau-specific initiatives. However, our review did not

address the GPRA requirement that the performance plan itself cover each program activity

contained in the budget, and we did not review whether the budgetary implications of the

plan itself were clear and transparent.

1 See the objectives, scope and methodology section towards the end of this letter for limitations in scope related to this second

objective.

2 Observations on the Department of Commerce’s Fiscal Year 1999 Annual Program Performance Report and Fiscal Year 2001

Annual Performance Plan (GAO/GGD-00-152R, June 30, 2000).
B-286123

Page 3 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

Based on our examination of its fiscal year 2001 plan, we believe Commerce has addressed

some, but not all, of the weaknesses we identified when examining the previous year’s plan.

Those weaknesses we identified were in the areas of strategies, resources, and performance

data verification and validation, as well as in performance measures.

This letter contains recommendations to the Secretary of Commerce to (1) include

explanatory information in its GPRA documents when significant changes are made to

agency performance goals, objectives or activities, performance measures and resource

requirements and (2) direct that Commerce agencies better link their sections of the GPRA

documents to congressional budget submissions and other budget documents.

In written comments on a draft of this letter, Commerce stated that the letter raised a number

of valid points that Commerce will consider in refining its next strategic plan and its future

annual performance plans and reports. Commerce also indicated that it will provide

explanations when major changes have been made to performance measures or goals. More

specific comments attributed to NOAA and TA included those agencies’ concurrence that

better linkages between performance data and budget data could be made. In addition, TA

raised two questions and stated that it would be pleased to contribute to any future

discussions on the questions that Congress and the administration might hold. The questions

asked whether agencies should be required to fully explain and document all changes to their

performance measures and whether measures should be developed and reported on for all

initiatives.

Background

The larger of Commerce’s two major science agencies is NOAA, whose fiscal year 2001

budget submission of $2.9 billion represented about 54 percent of the total Commerce budget

request. NOAA's mission is to describe and predict changes in the Earth's environment, as

well as conserve and manage coastal and marine resources to ensure sustainable economic

opportunities. Its components consist of the National Environmental Satellite, Data and

Information Service; the National Marine Fisheries Service; the National Ocean Service; the

National Weather Service; and Oceanic and Atmospheric Research. NOAA carries out its

mission by conducting research aimed at developing new technologies, improving operations,

and supplying the scientific basis for managing natural resources and solving environmental

problems. NOAA acquires observations from many sources, ranging from satellites and radar

to ships and submersibles. According to NOAA, these sources provide critical data and

quality information needed for the safe conduct of daily life and the basic functioning of a

modern society. NOAA’s products and services include short-term weather forecasts,

seasonal climate predictions, long-term global change prognoses, environmental

technologies, nautical charts, marine fisheries statistics and regulations, assessments of

environmental changes, and hazardous materials response information.

The smaller of the two science agencies, TA, prepared a congressional budget submission for

fiscal year 2001 of $722 million, or about 13 percent of the total Commerce budget request.

According to TA, its mission is to work with U.S. industry to maximize the technological
B-286123

Page 4 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

contribution to the nation's economy, serving as the executive branch focal point for civilian

technology and competitiveness issues. TA says that it fulfills its mission by maintaining and

improving key components of the technological infrastructure in the United States; fostering

the development, diffusion, and adoption of new technologies and leading business practices;

creating a business and policy environment conducive to innovation; and disseminating

technical information. TA is composed of the following: the National Institute of Standards

and Technology (NIST), the National Technical Information Service (NTIS), and the Office of

Technology Policy (OTP). TA also houses the Office of Air and Space Commercialization,

which coordinates Commerce’s efforts to foster commercial space development and

encourages private sector investment in space.

NOAA’s and TA’s Sections of the GPRA Plans and Report

Address Science Issues in a Generally Coherent Manner, but

Measuring Progress in Attaining Science-Related Performance

Goals Could Be Improved

The NOAA and TA sections of the performance report and performance plans address

science issues in a generally coherent and consistent manner. For example, NOAA’s and TA’s

mission statements and performance goals remained essentially unchanged in each of the

three GPRA documents. Further, NOAA and TA listed their performance goal strategies or

objectives, supporting activities, and crosscutting activities in the fiscal years 2000 and 2001

plans, and we noted that these descriptions also remained essentially unchanged. One

difference between the two plans was that an additional goal of protecting the nation’s

information infrastructure was added to TA’s section of the fiscal year 2001 plan. Also, both

NOAA and TA expanded their discussions on departmentwide or bureau-specific initiatives.

We did note that significant changes had been made to the performance measures and

resource requirements, and that these changes were not always explained in the GPRA

documents. Although GPRA guidance does not require that such changes be explained, in the

absence of explanations, the rationale for the performance measure and resource changes

that were made and the impact that they would have on addressing science issues was not

clear. We also noted that, in one case, no performance measures were established for a

performance goal. Additionally, performance measures were not always established for

NOAA’s or TA’s contribution to departmentwide initiatives, nor were measures established

for their respective bureau-specific initiatives. As a result, it will not always be easy to

determine the progress NOAA and TA make towards achieving their goals or contributing to

departmentwide efforts.

Performance Measures and Resources Were Changed Without Adequate

Explanation

We identified 10 performance measures established by NOAA or TA that had been dropped or

added with little or no explanation provided. We also noted instances in which--for specific
B-286123

Page 5 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

performance goals--the requested appropriation amounts identified in the fiscal year 2000

plan differed significantly from the enacted appropriation amounts identified in the fiscal

year 2001 plan. Yet, the fiscal year 2001 plan offered no explanations as to why this happened

or what impact the changed resource levels might have on the attaining of performance goals.

Finally, we identified one TA project that had been dropped from the subsequent year’s plan

with no explanation as to why. While the Office of Management and Budget’s (OMB) Circular

No. A-11, Part 2, which provides guidance for preparing GPRA documents, does not require

explanations for such changes, the lack of such an explanation in this case makes it more

difficult to understand what progress has been made on science issues.

In the fiscal year 2000 plan, NOAA’s goal of promoting safe navigation contained a

performance measure for the “Number of Physical Oceanographic Real-Time Systems

(PORTS) in Place to Provide Quality-Assured Data in Real-Time for Safe Navigation.”

According to NOAA, PORTS is an information system that, in part, furnishes real-time

navigational data to commercial and recreational mariners. In the plan, NOAA established a

performance target of increasing the number of these systems by three each year. This

measure was dropped from the fiscal year 2001 plan without explanation, and a new measure

for NOAA’s goal of safe navigation was added. The new measure relates to the completion of

the National Spatial Reference System (NSRS)--a system of reference stations and

monuments across the country--that is tied to the global positioning system (GPS). The NSRS

helps provide integrity to geographic coordinates obtained from GPS satellite signals used, in

part, for surveying, navigation, and mapping.

In another case, in the fiscal year 2001 plan, a measure--“Results of 90% of the research

activities cited in the 2001 IPCC Third Assessment of Climate Change”--was added for

NOAA’s performance goal of predicting and assessing decadal to centennial change.3

NOAA

provided no explanation as to why this measure was added. Further, we found the measure’s

title to be unclear, no explanation offered for why the 90 percent target had been selected,

and no discussion included on how the results of the measure’s research activities would be

used.

We also noted that, in some cases, changes were made from one plan to another in the

amounts of funding and FTE resources identified for specific performance goals. These

changes were not explained. For example, according to the fiscal year 2000 plan, NOAA

identified a total of $324 million and 1,661 FTEs needed for activities supporting its goal to

build sustainable fisheries. The fiscal year 2001 plan showed that NOAA actually received an

appropriation of $441.6 million and 2,371 FTEs for that effort. This represented increases of

$117.6 million (36 percent) and 710 FTEs (43 percent), yet no explanation for these

significant increases was provided in the plan to help the reader understand how the

increases affected the attaining of the goal.

In another example, according to the fiscal year 2000 plan, NOAA identified a total of $218

million and 608 FTEs needed for activities supporting its goal of sustaining healthy coasts.

3

The IPCC is the Intergovernmental Panel on Climate Change which periodically assesses scientific, technical, and economic

information in evaluating interferences in the climate system.
B-286123

Page 6 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

The fiscal year 2001 plan showed that NOAA actually received an appropriation of $278.7

million and 867 FTEs. This represented increases of $60.7 million (28 percent) and 259 FTEs

(43 percent), yet no explanation for these increases and how they affected performance

levels was provided. Furthermore, according to the fiscal year 2001 plan, NOAA identified an

additional increase in these amounts to $486.8 million and 912 FTEs for fiscal year 2001.

Again, no explanation for how these increases would affect performance was provided.

During a meeting with Commerce officials in which we discussed these two examples, we

were told that the differences between the appropriation amounts requested and enacted

were not as significant as they appeared. The officials said that the budget and FTE amounts

shown in the fiscal year 2000 plan--which purported to represent the total amounts required

for the respective performance goals--were erroneous in each case.4

They explained that the

budget and FTE amounts shown in the plan in each case represented appropriation requests

only for a NOAA component organization. In fact, according to NOAA officials, the amounts

portrayed in the plan as the total resource requirements needed for each of the performance

goals were understated because they did not include the budget and FTE requirements of the

other contributing Commerce organizations.

In the case of TA, according to the fiscal year 2000 plan, a project to foster the professional

development of mathematics and science teachers in elementary, middle, and secondary

schools was to begin as an activity supporting Commerce’s priority initiative of building upon

America’s technology infrastructure. According to the plan, although required FTEs were not

identified, the project was shown to require a $500,000 appropriation. The fiscal year 2001

plan did not include this project, nor did it explain why the project was dropped.

Performance Measures Not Always Established

According to Circular No. A-11, Part 2, performance measures must be included for

performance goals that are not self-measurable. In most cases, NOAA and TA complied with

this requirement. However, we did note one instance in which a performance goal was not

self-measurable, yet it had no performance measures. Commerce’s 2001 plan also identified

several departmentwide initiatives, and for most of those initiatives Commerce identified

bureau-specific performance measures. However, we noted one instance in which bureauspecific performance measures for a departmentwide initiative had not been established.

Finally, NOAA and TA identified their own bureau-specific initiatives, but did not identify

related performance measures. Circular No. A-11, Part 2, does not specifically discuss the

treatment of departmentwide or bureau-specific initiatives. However, establishing

performance measures for all such initiatives, or explaining how these initiatives relate to

existing performance goals and measures, could help the reader to learn whether progress

has been made in all performance areas.

4

According to NOAA officials, the correct total amounts that should have been shown in the fiscal year 2000 plan for the goal of

building sustainable fisheries were $426.3 million and 2,371 FTEs. For the goal of sustaining healthy coasts, they said the correct

total amounts that should have been shown in the plan were $315 million and 944 FTEs.
B-286123

Page 7 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

We noted that, in two cases, no performance measures had been established for either a TA

performance goal or a departmentwide initiative. Regarding the TA performance goal, we

mentioned earlier that a new TA performance goal of protecting the national information

infrastructure had been included in the fiscal year 2001 plan. No performance measure was

established for the goal. In the plan, TA explained that the goal was needed, in part, because

the ubiquitous and interconnected nature of information technology increases the extent to

which even limited attacks or failures can broadly disrupt the nation’s information

infrastructure. TA also identified the principal activity supporting this goal to be the

establishment and operation of the Institute for Information Infrastructure Protection (IIIP),

which will lead a partnership among industry, academia, and government to develop the R&D

capacity, technologies, and knowledge needed to protect the nation’s information

infrastructure. TA identified a measure for the goal entitled “Activity metrics related to

program establishment.” However, a close read of the discussion on that measure showed

that no measurable targets had been developed. TA explained that, as the IIIP becomes

established, it will build appropriate outcome measures into its long-term program and

operational plans. Although it may be premature to establish appropriate outcome measures

for attaining the performance goal, some intermediate output measures associated with

establishing the IIIP would have provided stakeholders a better sense of what TA expected to

accomplish in fiscal year 2001.

In the case of the departmentwide initiative, according to the fiscal year 2001 plan, there was

a priority initiative of “Addressing Critical Construction Needs.” This $33.8 million initiative

included construction and renovation of NOAA and TA facilities. However, unlike the other

departmentwide initiatives, no performance measures or targets were established for this

initiative. Appropriate output measures could have communicated NOAA’s and TA’s

intended fiscal year 2001 performance in this area.

In the case of bureau-specific initiatives, one of NOAA’s four such initiatives was entitled “the

Lands Legacy.” Under this $265.8 million initiative, according to the fiscal year 2001 plan,

NOAA was to--among other activities--provide resources to enhance national marine

sanctuaries and maintain and protect critical estuaries. No performance measures were

established for this or for NOAA’s other three bureau-specific initiatives. Similarly, TA, in the

fiscal year 2001 plan, identified three bureau-specific initiatives. For example, one was titled

“New Super-Fast Methods of Materials in Chemical Research.” This $4.5 million initiative

uses combinatorial methods to accelerate research, development, and testing of new

materials, ranging from pharmaceuticals to metal alloys to biological products. Again, no

performance measures were established for this or for TA’s other two bureau-specific

initiatives.
B-286123

Page 8 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

Goals and Resources in NOAA’s and TA’s Sections of the Fiscal

Year 2001 Plan Generally Matched Those in Their Sections of

the Fiscal Year 2001 Budget Submission, but Better Linkages

Can Be Made in the Future

Our review of the NOAA and TA sections of the fiscal year 2001 plan and their sections of the

fiscal year 2001 budget submission showed that, in most cases, the identified performance

goals, budgetary resources and FTE levels were the same. In other cases, however, they

differed to some degree. For example, we found direct linkages between the performance

plan and budget submission for NOAA’s performance goals and their supporting objectives or

activities. For TA, we found direct linkages for six of its seven performance goals, but not for

each goal’s supporting objectives or activities. And, for both NOAA and TA, there were no

linkages for any of their respective bureau-specific initiatives.

NOAA’s section of the performance plan provided budget and FTE information for each of its

seven performance goals, as well as budget information for each goal’s supporting objectives

or activities. We were able to easily track this information to NOAA’s section of Commerce’s

fiscal year 2001 budget submission. This was so because NOAA had included in its budget

submission a fiscal year 2001 resources summary table that (1) identified resource needs by

goals and objectives and (2) corresponded directly to the information presented in its section

of the performance plan. On the other hand, while TA’s section of the performance plan also

provided budget and FTE information for each of its seven performance goals, it did not

provide budget information for each goal’s supporting objectives or activities like NOAA’s

did. Therefore, we were unable to identify linkages to TA’s budget submission at the

objectives or activities level. At the performance goal level, although TA did not include in its

budget submission a summary table identifying resource requirements by goals and

objectives as NOAA did, we were nevertheless able to independently identify linkages to the

budget submission for six of TA’s seven performance goals. However, for one of the seven--

to assure and improve measurements and standards--we had to obtain the assistance of a TA

budget officer to determine how the fiscal year 2001 budgetary amount identified in the plan

for that goal linked to the fiscal year 2001 budget submission. The amount identified in the

plan for that goal was $368.2 million, while the amount identified in the budget submission

was $337.5 million. The budget officer explained that, because component activities for that

goal were identified in different places within the budget submission, calculations were

needed to reconcile the difference in the amounts.

Both NOAA and TA, in the performance plan, identified bureau-specific initiatives that were

listed and discussed separately from their performance goals and objectives. Budgetary

resources for these initiatives were not identified in the fiscal year 2001 plan, but were

identified in the fiscal year 2001 budget submission.
B-286123

Page 9 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

Not All Performance Plan Weaknesses Were Addressed by the

Fiscal Year 2001 Plan

In our review of Commerce’s fiscal year 2000 plan, we reported that the Department had

considerable success in improving its planning and the overall format and presentation of its

plan compared with the fiscal year 1999 plan.5

However, our review of the fiscal year 2000

plan also identified several areas in which improvements could still be made. Based on our

examination of its fiscal year 2001 plan, we believe Commerce addressed some, but not all, of

the weaknesses we had identified in the fiscal year 2000 plan.

In our report on the fiscal year 2000 plan, we identified general areas of weakness, including

those related to strategies, resources, and performance data verification and validation. We

also identified weaknesses in the area of performance goals and measures, and earlier in this

letter we highlighted some of the problems that still exist in that area. The following

information describes in more detail the other weaknesses we identified and the extent to

which they have been addressed in the NOAA and TA sections of Commerce’s fiscal year 2001

plan.

Strategies and Resources

In our report on the fiscal year 2000 plan, we stated that the plan’s descriptions of strategies

and resources were generally vague and did not clearly relate to the achievement of identified

performance goals and targets. Our review of the fiscal year 2001 plan showed that, for the

NOAA and TA sections of the plan, Commerce has improved its descriptions of strategies as

well as resource requirements.

We previously stated that the strategies and resources associated with performance goals in

the fiscal year 2000 plan were, in some cases, general and not tailored specifically for the

achievement of these goals. In addition, we stated that the fiscal year 2000 plan did not

clearly show how funding, FTE, and information technology (IT) resources would be

allocated to performance goals. Further, we said the plan did not generally discuss how

external factors that could affect targeted performance levels would be mitigated.

In the fiscal year 2001 plan, the Department used largely the same language to identify the

strategies for NOAA’s and TA’s performance goals as was used in the fiscal year 2000 plan.

However, as discussed earlier, one improvement was that the NOAA section of the fiscal year

2001 plan related a specifically identified budget amount to each strategy. Our review of the

fiscal year 2001 plan also showed that, for NOAA and TA, Commerce clearly listed budget,

FTE, and IT resource requirements for 13 of these two bureaus’ 14 performance goals. The

only exception was that TA did not delineate an IT resource requirement for its goal to

protect the National Information Infrastructure.

5 Observations on the Department of Commerce’s Fiscal Year 2000 Performance Plan (GAO/GGD-99-117R, July 20, 1999).
B-286123

Page 10 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

Regarding mitigation strategies for addressing external factors that could affect targeted

performance, the fiscal year 2001 plan offered some improvement over the fiscal year 2000

plan. But the 2001 plan, like the 2000 plan, did not provide mitigation strategies for external

factors affecting some goals. While the NOAA and TA sections of the fiscal year 2000 plan did

not identify strategies for addressing external factors, these sections of the fiscal year 2001

plan did identify mitigation strategies for external factors for 5 of 14 goals. However, the plan

did not provide mitigation strategies to address external factors listed for the other nine

performance goals. For example, under its performance goal of sustaining healthy coasts,

NOAA identified several coastal stewardship activities that depend on the contributions of

states, territories, and federal agencies. While it recognized that the failure of one or more of

these partners to fulfill their contributions to these activities could seriously affect the effort

to attain the performance goal, NOAA did not provide a mitigation strategy for addressing this

factor. In addition, under TA’s performance goal of analyzing and developing technology

policies, it identified the objective of coordinating and leading state and federal partnership

efforts for key interagency technology programs. In the plan, TA identified as an external

factor the fact that outputs associated with these partnership programs depend on the

interest and commitment of the participants. However, TA did not identify a strategy for

addressing that factor.

Performance Data Verification and Validation

We previously reported that the fiscal year 2000 plan did not adequately (1) explain how

Commerce bureaus would verify and validate data, (2) identify sources of the data, and (3)

identify limitations of the data for specific goals and measures. Verification refers to the

accuracy of the data, and validation refers to the appropriateness of the data for measuring

performance. Accurate and valid information is important for measuring progress towards

achieving performance goals and for making informed program decisions. Our review of the

fiscal year 2001 plan showed that, for NOAA and TA, efforts were made to improve data

verification and validation plans and to identify the sources and limitations of the data.

Nevertheless, in some cases, NOAA and TA could further describe the sources and validity of

the data used to measure progress in attaining their performance goals.

In the fiscal year 2001 plan, NOAA and TA, in some cases, provided expanded explanations in

the data validation and verification sections of the plan. For example, in the case of NOAA’s

goal of providing advance short-term warnings and forecasts--specifically, the related

measure to increase lead time and accuracy of tornado warnings--the fiscal year 2001 plan

provided an expanded explanation of how data used to measure performance would be

collected, verified, and reported. The plan also indicated that the frequency of collecting and

reporting those data would be increased from an annual basis to a monthly basis. However,

there was no information provided on the validity of the data for this performance measure.

The language used to describe data limitations remained unchanged from the prior year’s

plan.

In another case, for TA’s goal to assure and improve measurements and standards--

specifically, the related measure of standard reference materials available--the data
B-286123

Page 11 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

verification and validation section of the fiscal year 2001 plan provided an expanded

discussion of data verification and highlighted the data limitations. Similarly expanded

information was provided for some of TA’s other goals and measures as well. Nevertheless,

little explanation was generally provided for the sources of the data, and, as with NOAA,

information regarding the validity of the data was lacking.

Conclusions

The NOAA and TA sections of the fiscal year 1999 performance report and fiscal years 2000

and 2001 plans tell a generally coherent story in addressing science issues in that their

missions and performance goals were consistently stated in each document. However, since

the issuance of the fiscal year 2000 plan, significant, unexplained changes have been made to

NOAA’s and TA’s performance measures and identified resource needs. While explanations

for these significant changes are not required by either GPRA or by OMB, we believe such

explanations would provide a more complete picture of what the agencies are trying to

accomplish and how they plan to do so. Further, the exercise of developing explanations for

such changes, we believe, would help identify the kinds of errors we encountered when

comparing data from one plan to another. Also, we believe opportunities exist for

establishing performance measures for departmentwide and bureau-specific initiatives. The

explanations for changes and the added performance measures would be useful to Congress

as it carries out its oversight and appropriation responsibilities, to internal and external

auditors as they evaluate agency programs and activities, and to agency managers and line

personnel as they work together to effectively implement agency programs and activities.

Further, we believe that better linkages, i.e., a closer match, between information in the

GPRA documents and in the NOAA and TA sections of the budget submission would help

provide users with perspective in terms of the size and significance of the goals, objectives, or

activities, especially as the goals, objectives, or activities evolve or change over time. Also,

we believe that identifying budgetary resources for departmentwide initiatives in the annual

performance plans would provide stronger linkages between the plan and the budget

submission. In addition, we believe that linkages could be improved for TA in the future if it

were to identify budgetary resources for its individual objectives or activities and include a

corresponding summary table in its budget submission.

Finally, we believe that continuing efforts by Commerce to address weaknesses we identified

when examining the fiscal year 2000 performance plan, especially those related to

performance data verification and validation, would help increase confidence in the results

reported in the future.
B-286123

Page 12 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

Recommendations

We recommend that the Secretary of Commerce (1) include in the GPRA documents

explanatory information related to significant changes made to agency performance goals,

objectives or activities, performance measures, and resource requirements and (2) direct that

Commerce agencies better link their sections of the GPRA documents to congressional

budget submissions and other budget documents by identifying budgetary resources at the

bureau-specific initiative levels and the performance goal objective or activity levels in both

the GPRA and budget documents.

Agency Comments and Our Evaluation

On September 20, 2000, the Secretary of Commerce submitted to us written comments on a

draft version of this letter. Those comments included Commerce’s general observations on

our findings and recommendations as well as specific comments from NOAA and TA. Some

of the specific comments included questions about implementing GPRA. We have presented

those questions below for consideration by those involved in implementing GPRA and

utilizing GPRA-related documents.

Commerce’s general observations were that our letter raised a number of valid points that

Commerce will consider in refining its next strategic plan and its future annual performance

plans and reports. In addition, Commerce indicated that it will implement at least part of our

recommendation by providing explanations when major changes have been made to

performance measures or goals. Commerce did not indicate, in its general observations,

whether or not it agreed with the other part of our recommendation--that is, the need to

better link sections of the GPRA documents to congressional budget submissions and other

budget documents.

Regarding NOAA’s and TA’s specific comments, both concurred that better linkages between

performance data and budget data could be made. For example, NOAA stated that our

finding that budget initiatives (e.g., the $265.8 million bureau-specific Lands Legacy initiative)

and performance measures could be more explicitly linked was valid. In fact, NOAA stated

that it concurred with our recommendation that called for a better linkage between the GPRA

documents and the congressional budget submissions and other budget documents by

identifying budgetary resources at the bureau-specific initiative levels. TA stated that the

linkage between its budget data, contained in both the performance plan and the budget

submission, could be better articulated and that its future budget submissions will include a

table that aligns its budget data with the structure of the annual performance plan.

NOAA and TA raised some concerns about other matters covered in our draft. For example,

in response to our concern about significant, unexplained differences between appropriation

amounts requested and amounts actually provided, NOAA stated that “...funds requested by

the administration are not always appropriated by Congress, and explanations are not
B-286123

Page 13 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

necessarily provided, so it would not be productive or prudent to speculate on this aspect.”

We agree that it would be inappropriate to speculate on why funding differences occurred.

However, because significant funding differences can materially affect the attainment of

performance goals, we believe that it would be beneficial to provide the reader with the

rationale for the funding differences, if available, as well as an explanation of how the funding

differences will affect attainment of the goals.

NOAA further indicated that the impact of changed resource levels would be more

appropriately addressed in the annual performance report as opposed to the annual

performance plan. We agree that “actual impact” from the significantly changed resource

levels would be appropriately addressed in the subsequent performance report. However, we

also believe that information regarding planned or “expected impact” from such resource

changes would be appropriately included in the performance plan, since the available

resources directly affect planned attainment of performance goals.

In its specific comments, TA raised two questions and stated that it would be pleased to

contribute to any future discussions that GPRA leaders in Congress and the administration

might hold on these two questions. The first question was: “Should agencies be required to

fully explain and document all changes to their performance measures in any given

performance plan and corresponding performance report, or should this type of information

be conveyed through other channels (for instance, through stakeholder consultations)?” The

second question was: “Should agencies be required to provide and subsequently report on

performance measures for all initiatives, or should GPRA performance plans and reports,

focus on the ‘vital few’ measures that best characterize performance in core programmatic

areas?”

Finally, TA commented that GPRA remains unclear on the extent to which agencies should

formally document anticipated performance levels associated with all initiatives, regardless

of level, scope, and character. TA wondered whether a $500,000 initiative should receive the

same level of metric development and reporting as a $40 million initiative and whether

performance measures for construction initiatives should be developed on a par with

measures for new and/or existing research programs.

Objectives, Scope, and Methodology

The objectives of our work were to determine (1) whether the NOAA and TA sections of

Commerce’s fiscal year 1999 performance report and fiscal years 2000 and 2001 performance

plans told a coherent story in addressing science issues, (2) whether NOAA’s and TA’s

performance goals, budgetary and FTE resource requirements identified in Commerce’s fiscal

year 2001 performance plan linked to NOAA’s and TA’s sections of Commerce’s fiscal year

2001 congressional budget submission, and (3) whether the NOAA and TA sections of

Commerce’s fiscal year 2001 performance plan addressed the weaknesses that we identified

in our review of Commerce’s fiscal year 2000 performance plan.

The scope of our work was limited to the two Commerce science-related agencies: NOAA and

TA. The fiscal year 2001 budget requests for these two science-related agencies, together,
B-286123

Page 14 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

accounted for approximately 67 percent of Commerce’s overall fiscal year 2001 budget

request.

Concerning our second objective, GPRA requires “...each agency to prepare an annual

performance plan covering each program activity set forth in the budget of such agency.”

Circular No. A-11, Part 2, further requires that, “Each program activity in an agency’s Program

and Financing Schedules in the Budget Appendix must be covered by a performance goal or

indicator in the annual performance plan.” The scope of this review did not include

examining compliance with those requirements. A review of those requirements is included

in the scope of another, ongoing, GAO review. The results of that review should be reported

in the near future.

Our methodology included a review of information contained in Commerce’s fiscal year 1999

performance report and fiscal years 2000 and 2001 performance plans related to the two

science agencies. This information included the agencies’ mission statements, priorities,

performance goals and measures, and resource requirements. We also examined

Commerce’s fiscal year 2001 budget submission to Congress for the two science-related

agencies. We evaluated the extent to which there were linkages between those agencies’

resource requirements, as identified in their sections of Commerce’s fiscal year 2001

performance plan, and the information contained in their sections of Commerce’s fiscal year

2001 congressional budget submission. In addition, to determine the extent to which NOAA’s

and TA’s sections of Commerce’s fiscal year 2001 plan addressed weaknesses we had

identified in the prior year’s plan, we first identified those weaknesses by examining our prior

report on Commerce’s fiscal year 2000 performance plan. We then examined the NOAA and

TA sections of Commerce’s fiscal year 2001 performance plan to determine the extent to

which they addressed those weaknesses. Finally, we submitted written questions to, and

interviewed, appropriate Commerce officials to obtain answers and explanatory information

regarding questions that arose during our review.

We did not independently verify the information contained in the performance report or

plans. We conducted our review in July and August 2000 in accordance with generally

accepted government auditing standards.

As arranged with your office, unless you publicly announce its contents earlier, we plan no

further distribution of this letter until 30 days from the date of this letter. At that time, we

will send copies of this letter to Representatives Vernon J. Ehlers and Ralph M. Hall in their

capacities as Vice Chairman and Ranking Minority Member, respectively, of the House

Science Committee. We will also, at that time, send copies to Representatives Harold Rogers

and Jose E. Serrano in their capacities as Chairman and Ranking Minority Member,

respectively, of the House Appropriations Subcommittee on Commerce, Justice, State and

Judiciary. And, at that time, we will send copies to Representatives Dan Burton and Henry

Waxman in their capacities as Chairman and Ranking Minority Member, respectively, of the

House Committee on Government Reform.

Also, at that time, we will send copies to Senators Judd Gregg and Ernest Hollings in their

capacities as Chairman and Ranking Minority Member, respectively, of the Senate
B-286123

Page 15 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies

Appropriations Subcommittee on Commerce, Justice, State and the Judiciary; to Senator

John McCain in his capacity as Chairman of the Senate Commerce, Science, and

Transportation Committee; and to Senators Fred Thompson and Joseph Lieberman in their

capacities as Chairman and Ranking Minority Member, respectively, of the Senate

Governmental Affairs Committee. At that time, we will also send copies to the Honorable

Jacob J. Lew, Director of OMB, and we will publish this letter on our Internet Web site,

www.gao.gov

If you have any questions about this report, please contact me or N. Scott Einhorn on

(202) 512-8676. Key contributors to this assignment were Gerard Burke and

Jeremy Latimer.

Sincerely yours,

Michael Brostek

Associate Director, Federal Management

And Workforce Issues
Page 16 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies
Page 17 GAO/GGD-00-197R GPRA Plans For Commerce’s Science Agencies
Ordering Copies of GAO Reports

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order made

out to the Superintendent of Documents, when necessary. VISA

and MasterCard credit cards are accepted, also. Orders for 100 or

more copies to be mailed to a single address are discounted 25

percent.

Order by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 or by using

fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to

obtain these lists.

Viewing GAO Reports on the Internet

For information on how to access GAO reports on the INTERNET,

send e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

Reporting Fraud, Waste, and Abuse in Federal Programs

To contact GAO FraudNET use:

Web site: http://www.gao.gov/fraudnet/fraudnet.htm

E-Mail: fraudnet@gao.gov

Telephone: 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100

(410598)