United States Government Accountability Office

GAO Report to Congressional Requesters

WEATHER 

FORECASTING 

National Weather 

Service’s Operations 

Prototype Needs More 

Rigorous Planning 

June 2007 

 

GAO-07-650 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

June 2007

WEATHER FORECASTING 

National Weather Service’s Operations 

Highlights of GAO-07-650, a report to Prototype Needs More Rigorous Planning 

congressional requesters 

Using advanced systems and 

trained specialists located in 122

weather forecast offices 

throughout the country, the

National Weather Service (NWS) 

provides storm and flood warnings

and weather forecasts to protect

life and property and to enhance 

the national economy. To improve 

the efficiency of its operations, in 

November 2006, NWS approved an

effort to develop a prototype of an 

alternative way of operating. Under 

this prototype, weather forecasting

offices would share selected 

responsibilities. 

GAO (1) determined the status of

and plans for the prototype, 

(2) evaluated whether the 

prototype’s justification was 

sufficient, (3) determined whether 

NWS’s plans to evaluate the

prototype are adequate, 

(4) evaluated whether NWS is

sufficiently involving stakeholders 

in its prototype plans, and 

(5) determined how NWS plans to 

ensure that there will be no 

degradation of service during and 

after the prototype. To do so, GAO 

analyzed agency documentation 

and interviewed program officials 

and stakeholders. 

NWS’s prototype is currently on hold pending a reevaluation of the agency’s 

approach. The agency had just begun the first phase of its three-phased 

prototype to demonstrate a new concept of operations over a 2-year period 

when, in late March 2007, the Department of Commerce’s Under Secretary 

suspended the prototype because of concerns about the agency’s approach. 

In the first phase, NWS established a program manager and began planning 

for the next two phases. During the remaining phases, NWS planned to have 

20 weather forecasting offices share responsibilities in 2-office pairs and 

then in 4-office clusters. NWS then planned to decide whether to implement 

the new concept of operations on a national basis. 

The justification for the prototype was not sufficient. Before the prototype 

was suspended, the agency had approved moving forward with its prototype 

without conducting a cost-benefit analysis. NWS estimated that the 

prototype would cost approximately $9.3 million and would offer qualitative 

benefits, such as increased efficiency and an improved ability to focus on 

severe weather events, but did not quantify benefits or the expected return 

on its investment. If NWS were to proceed with the prototype without a costbenefit analysis, it would lack assurance that its approach would be a costeffective investment for the agency. 

NWS identified goals and selected measures to evaluate during its prototype 

activities, but it did not establish a rigorous evaluation plan. Specifically,

NWS did not define a full set of needed measures, how it planned to compare 

prototype results with baseline performance, or how its selected measures 

supported the prototype goals. If NWS were to proceed without a rigorous 

evaluation plan, the agency would run an increased risk of not sufficiently 

measuring the impact of changes on its performance and could make 

decisions affecting the nation’s weather on the basis of incomplete or flawed

data. 

Although NWS involved internal stakeholders in planning its prototype, it did 

not (1) involve external stakeholders or (2) establish a plan that identified 

key stakeholders, the stakeholders’ responsibilities, and a time line for 

involving stakeholders and addressing their comments. If NWS were to 

proceed with its prototype without such a plan, it could not ensure that 

stakeholder interests would be identified and addressed. 

NWS planned to mitigate the risk of degradation during and after the 

prototype by conducting laboratory exercises to understand the impact of 

the prototype, using a dedicated workstation in each office so that the office 

could switch to original systems if warranted, and monitoring its systems 

and products during the prototype. NWS officials stated that should the 

agency decide to implement the prototype on a national basis, it has 

standard procedures for testing and validating systems and software to avoid 

any degradation of service. 

What GAO Recommends

GAO is recommending that if NWS

decides to proceed with its

prototype, it undertake a more 

rigorous approach to justifying, 

evaluating, and involving stakeholders in the prototype. In written 

comments, Commerce agreed with 

GAO’s recommendations. 

www.gao.gov/cgi-bin/getrpt?GAO-07-650.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact David Powner 

at (202) 512-9286 or pownerd@gao.gov. 
Contents 

Letter 1

Results in Brief 2

Background 4

NWS Initiated a 2-Year Prototype and Then Suspended It 14 

NWS Has Not Yet Sufficiently Justified Its Prototype 16 

NWS Lacks a Rigorous Evaluation Plan for Assessing Its Prototype 17 

NWS Involved Internal Stakeholders in Planning the Prototype, but 

Did Not Involve External Stakeholders or Establish a Plan for 

Stakeholder Involvement 19 

NWS Planned to Ensure That the Prototype Would Not Degrade Its 

Service 21

Conclusions 21

Recommendations for Executive Action 22 

Agency Comments and Our Evaluation 22 

Appendix I Objectives, Scope, and Methodology 24 

Appendix II NWS Prototype Plans, as of March 2007 27 

Appendix III Comments from the Department of Commerce 29 

Appendix IV GAO Contact and Staff Acknowledgments 32 

Table 

Table 1: Clustered Peer Concept of Operations Prototype Costs, as 

of March 2007 17 

Figures 

Figure 1: NWS’s 122 Weather Forecast Offices 6 

Figure 2: Overview of Key Systems and Technologies Supporting 

NWS Forecasts 7 

Figure 3: An AWIPS Workstation 9 

Page i GAO-07-650 Weather Forecasting
Figure 4: NWS Offices Involved in the Clustered Peer Concept of 

Operations Prototype 14 

Figure 5: Preliminary Schedule for NWS’s Clustered Peer Concept 

of Operations Prototype, as of November 14, 2006 15 

Abbreviations 

AWIPS Advanced Weather Interactive Processing System 

FAA Federal Aviation Administration 

NOAA National Oceanic and Atmospheric Administration

NWS National Weather Service 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page ii GAO-07-650 Weather Forecasting
United States Government Accountability Office

Washington, DC 20548 

June 8, 2007

The Honorable Nick Lampson 

Chairman 

The Honorable Bob Inglis

Ranking Member 

Subcommittee on Energy and Environment 

Committee on Science and Technology 

House of Representatives 

The Honorable David Wu 

House of Representatives 

The Honorable Vernon J. Ehlers 

House of Representatives 

The National Weather Service’s (NWS) ability to forecast the weather 

affects the life and property of every American. The agency’s basic mission 

is to provide storm and flood warnings and weather forecasts for the 

United States, its territories, and adjacent oceans and waters, in order to 

protect life and property and to enhance the national economy. NWS also 

supports other federal agencies’ operations by providing aviation- and 

marine-related weather forecasts and warnings. To fulfill its mission, NWS 

operates 122 weather forecasting offices and other specialized weather 

centers across the country. In these offices, trained meteorologists use 

advanced systems to provide forecasts and warnings for local geographic 

areas. 

In an effort to improve its operational efficiency, in August 2005, NWS 

chartered a working group to evaluate the roles, responsibilities, 

functions, and supporting technology of weather forecast offices 

nationwide and to make a proposal for a more efficient concept of 

operations. In December 2005, the working group proposed an alternative 

way of operating in which weather offices could share some weather 

forecasting functions—particularly during high-intensity weather events. 

After receiving this proposal, NWS chartered another team to develop

plans for a prototype of this concept to more fully evaluate this approach.

Because of your interest in weather service operations, we (1) determined 

the current status of and plans for the prototype, (2) evaluated whether 

the justification for the prototype was sufficient, (3) determined whether 

Page 1 GAO-07-650 Weather Forecasting
NWS’s plans to evaluate the prototype are adequate, (4) evaluated whether 

NWS is sufficiently involving stakeholders in its prototype plans, and 

(5) determined how NWS plans to ensure that there will be no degradation 

of service during and after the prototype. 

In late March 2007, after our audit work was nearly completed, the 

Department of Commerce’s Under Secretary for Oceans and Atmosphere

directed the Director of the National Weather Service to place all activities

associated with the agency’s concept of operations on hold pending a 

review by the Deputy Under Secretary. NWS officials stated that the 

prototype was suspended because of concerns raised about the agency’s 

approach by some Members of Congress and by National Oceanic and 

Atmospheric Administration (NOAA) leadership, the private sector, NWS’s 

workforce, and its critical emergency management partners. Shortly after 

this decision was made, we met with your staffs and agreed to continue 

with plans to issue this report to offer insight and recommendations to 

NWS, should it decide to proceed with the prototype. 

To address our objectives, we reviewed prototype plans and presentations. 

We compared program documents with best practices for justifying and

evaluating prototypes and for involving stakeholders. We also interviewed 

internal and external stakeholders and agency officials. We conducted our 

work at NWS headquarters in the Washington, D.C., metropolitan area;

NWS offices in Tampa, Florida, Detroit, Michigan, and Kansas City, 

Missouri; and a NOAA office in Boulder, Colorado, because of the 

relevance of these sites to the prototype. We performed our work from 

October 2006 to April 2007, in accordance with generally accepted 

government auditing standards. Additional details on our objectives, 

scope, and methodology are provided in appendix I. 

NWS had begun the first phase of its three-phased prototype to

demonstrate a new concept of operations in which weather offices would 

share selected forecasting responsibilities, when the Under Secretary for 

Oceans and Atmosphere suspended the prototype because of concerns 

about the agency’s approach. As of March 2007, NWS had established a 

program manager and begun to document how the prototype would be 

executed and to develop training for employees who would participate.

During the second and third phases, NWS planned to have 20 weather 

forecasting offices share forecasting responsibilities in 2-office pairs and 

then in 4-office clusters. At the conclusion of the 2-year prototype effort, 

NWS planned to make a decision on whether to proceed to implement the 

new concept of operations on a national basis. However, these efforts 

Results in Brief 

Page 2 GAO-07-650 Weather Forecasting
were halted when the agency suspended the prototype pending a 

reevaluation on how best to proceed.

Before the prototype was suspended, NWS had approved moving forward 

with its prototype plan without conducting an analysis of the relative costs 

and benefits of the initiative. NWS estimated that the prototype would cost 

approximately $9.3 million and identified expected qualitative benefits—

including increased efficiency, improved ability to focus on high-impact 

weather events, and the ability to switch to a backup site more quickly 

when a weather forecast office is disabled due to loss of power or 

communications. However, the agency has not conducted a cost-benefit 

analysis or determined whether there would be a return on investment for 

the prototype. If NWS were to proceed with the prototype without a costbenefit analysis, it would lack assurance that its approach would be a costeffective investment for the agency. 

NWS identified goals and selected measures to evaluate during its 

prototype activities, but it did not establish a rigorous evaluation plan. 

NWS’s prototype plan identified evaluation goals, such as to validate that a 

group of weather forecast offices, operating as one unit, could be more 

effective and efficient than each office on its own. Also, the prototype plan 

identified quantitative and qualitative measures to be used in evaluating

the prototype, including the number of times that work was transferred 

between offices to focus on high-impact events, the number of additional 

hours made available during benign weather for nonforecast work, and 

customer perceptions of service quality and timeliness. However, NWS did 

not establish an evaluation plan that identified a comprehensive set of 

applicable measures, what baseline performance its prototype would be 

compared with, and how its measures supported the goals of the 

prototype. If NWS were to proceed with its prototype without a rigorous 

evaluation plan, it would run an increased risk of not sufficiently 

measuring the impact of planned changes on its performance, and of 

subsequently making decisions affecting the nation’s weather service on 

the basis of incomplete and flawed data. 

NWS involved internal stakeholders in planning its prototype, including 

having selected internal stakeholders review plans and provide feedback 

on work products and laboratory activities. However, NWS did not involve 

external stakeholders in its planning for the prototype or establish a plan

for stakeholder involvement during the prototype. Specifically, NWS did 

not request input from external stakeholders, including emergency 

managers, in the development and planning of the prototype. In addition, 

the agency did not establish a plan that identified key external

Page 3 GAO-07-650 Weather Forecasting
stakeholders or stakeholder representatives, determined the 

responsibilities of internal or external stakeholders, and established a time 

line for involving stakeholders and addressing their comments. If NWS 

were to proceed with its prototype without such a plan, the agency would 

be unable to ensure that employee and customer interests are anticipated, 

identified, and addressed. 

Prior to suspending the prototype, NWS stated that it would ensure that 

there would be no degradation of service resulting from the prototype, and 

that it had plans in place to mitigate the risk of degradation. To these ends, 

NWS planned to conduct multiple laboratory exercises to understand the 

impact of the prototype. Also, NWS planned to conduct the prototype on a 

dedicated workstation in each cluster office that was separate from 

existing systems, so that it could revert to normal operations should the 

need arise. In addition, NWS planned to monitor its systems and products

under the prototype, so that the agency could revert to baseline systems if 

warranted. NWS officials stated that, should the agency decide to 

implement the prototype on a national basis, it has standard procedures 

for testing and validating systems and software to avoid any degradation 

of service before they are put into use. 

We are making recommendations to the Secretary of Commerce to ensure

that if NWS decides to proceed with its prototype, it undertake a more 

rigorous approach to justifying, evaluating, and involving stakeholders in 

its prototype effort. We also are recommending that the agency evaluate

moving forward with technology upgrades to allow offices to switch to 

backup service more quickly during high-impact weather events. 

Commerce provided written comments on a draft of this report in which it 

agreed with our recommendations, and stated that our findings were 

included in the Under Secretary’s recent review of the concept of 

operations prototype. Based on that review, the NOAA Administrator 

directed NWS to cease all activities associated with the concept of 

operations prototype, and to undertake a comprehensive and analytical 

review to determine new and revised requirements for NWS products and 

services. A reprint of the department’s comments is provided in appendix 

III. 

The mission of NWS—an agency within Commerce’s NOAA—is to provide 

weather, water, and climate forecasts and warnings for the United States,

its territories, and its adjacent waters and oceans to protect life and 

property and to enhance the national economy. NWS is the official source 

Background 

Page 4 GAO-07-650 Weather Forecasting
of aviation- and marine-related weather forecasts and warnings as well as

warnings about life-threatening weather situations. 

NWS Office Structure: An 

Overview

The coordinated activities of weather facilities throughout the United 

States allow NWS to deliver a broad spectrum of climate, weather, water, 

and space weather services. These facilities include weather forecast 

offices, river forecast centers, national centers, and aviation center 

weather service units. The functions of these facilities are described in the 

following text: 

• 122 weather forecast offices are responsible for providing a wide variety of

weather, water, and climate services for their local county warning areas, 

including advisories, warnings, and forecasts. See figure 1 for a map of the 

locations of these offices. 

• 13 river forecast centers provide river, stream, and reservoir information 

to a wide variety of government and commercial users as well as to local

weather forecast offices for use in flood forecasts and warnings. 

• 9 national centers constitute the National Centers for Environmental 

Prediction, which provide nationwide computer model output and manual 

forecast information to all NWS field offices and to a wide variety of 

government and commercial users. These centers include the 

Environmental Modeling Center, Storm Prediction Center, Tropical 

Prediction Center/National Hurricane Center, Climate Prediction Center, 

Aviation Weather Center, and Space Environment Center, among others.

• 21 aviation center weather service units, which are colocated with key 

Federal Aviation Administration (FAA) air traffic control centers across 

the nation, provide meteorological support to air traffic controllers. 

Page 5 GAO-07-650 Weather Forecasting
Figure 1: NWS’s 122 Weather Forecast Offices 

Sources: NWS (data) and Map Resources (map).

Seattle

Portland Pendleton

Spokane

Missoula

Great Falls

Medford

Eureka

Sacramento Reno

Elko

San Francisco

Hanford

Las Vegas

Los Angeles

San Diego

Flagstaff Albuquerque

El Paso

Amarillo

Lubbock

Midland

Phoenix

Tuscon

Glasgow Grand Forks

Bismarck

Aberdeen

Rapid City Sioux Falls

Billings

Riverton

Cheyenne

Boise

Pocatello

Salt Lake City

San Angelo

San Antonio

Fort Worth

Houston

Corpus Christi

Brownsville

Denver

Grand Junction

Pueblo

Goodland

Topeka

Wichita

Tulsa

Norman

Dodge City

North Platte Omaha

Des Moines

Davenport

Minneapolis

Hastings

Duluth

La Crosse

Green Bay

Marquette

Milwaukee

Gaylord

Grand Rapids

Detroit

Cleveland

Wilmington

Northern 

Indiana

Indianapolis

Louisville

Jackson

Chicago

Lincoln

St. Louis

Kansas City

Springfield

Little Rock

Shreveport

Lake Charles

New Orleans

Jackson

Mobile

Birmingham

Tallahassee

Jacksonville

Melbourne

Miami

Key West

Tampa

Atlanta

Huntsville

Nashville Knoxville

Memphis

Charleston

Wilmington

Newport Raleigh

Columbia

Greer

Paducah Blacksburg

Wakefield

Sterling

Charleston

Pittsburgh

State College

Buffalo

Philadelphia

Binghamton

Albany

Burlington

New York City

Boston

Portland

Caribou

Honolulu

Anchorage

Fairbanks

Juneau

San Juan

Guam Alaska Hawaii Puerto Rico

Weather Forecast Office

Page 6 GAO-07-650 Weather Forecasting
To fulfill its mission, NWS relies on a national infrastructure of systems 

and technologies that gather and process data from the land, sea, and air. 

NWS collects data from many sources, including ground-based Automated 

Surface Observing Systems, the Next Generation Weather Radars, and 

operational environmental satellites. These data are integrated by 

advanced data processing workstations—called an Advanced Weather 

Interactive Processing System (AWIPS)—and used by meteorologists to 

issue local forecasts and warnings. Also, these data are fed into

sophisticated computer models running on high-speed supercomputers, 

which are then used to help develop forecasts and warnings. Figure 2 

depicts the integration of the various systems and technologies and is 

followed by a description of each. 

NWS Relies on Key 

Systems and Technologies 

to Fulfill Its Mission 

Figure 2: Overview of Key Systems and Technologies Supporting NWS Forecasts 

Source: GAO.

Next Generation 

Weather Radars

Operational 

environmental 

satellites

Automated Surface 

Observing Systems

Advanced Weather Interactive 

Processing Systems

Numerical weather 

models run on 

supercomputers

The Automated Surface Observing System is a system of sensors, 

computers, display units, and communications equipment that automates 

the ground-based observation and dissemination of weather information 

nationwide. This system collects data on temperature, dew point, visibility, 

Automated Surface Observing 

System 

Page 7 GAO-07-650 Weather Forecasting
wind direction and speed, pressure, cloud height and amount, and types 

and amounts of precipitation. There are currently 1,001 units deployed 

across the United States, with NWS, FAA, and the Department of Defense 

operating 312, 571, and 118 units, respectively. 

The Next Generation Weather Radar is a Doppler radar system that 

detects, tracks, and determines the intensity of storms and other areas of 

precipitation; determines wind velocities in and around detected storm 

events; and generates data and imagery to help forecasters distinguish 

hazards, such as severe thunderstorms and tornadoes. The radar system 

also provides information about heavy precipitation that leads to warnings 

about flash floods and heavy snow. A network of these radars is made up 

of 158 operational radar systems and 8 nonoperational systems that are 

used for training and testing. Of these, NWS, the United States Air Force, 

and FAA sponsor 120, 26, and 12 radars, respectively. 

Next Generation Weather 

Radar 

Although NWS does not own or operate satellites, geostationary and polarorbiting environmental satellites1

 are key sources of data for its 

operations. These satellite systems continuously collect environmental 

data about Earth’s atmosphere, surface, and cloud cover and the 

electromagnetic environment. These data are used by meteorologists to 

develop weather forecasts and other services, and are critical to the early 

and reliable prediction of severe storms, such as tornadoes and 

hurricanes. 

Operational Environmental 

Satellites 

AWIPS is a data processing workstation that integrates and displays 

hydrometeorological data at NWS weather forecast offices, river forecast 

centers, and national centers. This system integrates data from a variety of 

sources (including the ground systems, radars, and satellites previously 

listed) to produce rich graphical displays to aid forecaster analysis and 

decision making. AWIPS includes a graphical forecast editing tool that 

allows forecasters to display and manipulate detailed graphical depictions 

of expected weather, and to use these depictions to generate text and 

graphical forecasts. AWIPS is used to disseminate weather information to 

the national centers; weather offices; the media; and other federal, state, 

and local government agencies. Figure 3 shows a standard AWIPS 

workstation. 

Advanced Weather Interactive 

Processing System 

 1

Geostationary environmental satellites orbit above Earth’s surface at the same speed as 

Earth rotates, so that each satellite remains over the same location on Earth. Polar 

satellites constantly circle Earth in an almost north-south orbit, providing global coverage

of conditions that affect the weather and climate. 

Page 8 GAO-07-650 Weather Forecasting
Figure 3: An AWIPS Workstation

Source: NOAA.

Numerical weather models are advanced software programs that 

assimilate data from satellites and ground-based observing systems and 

provide short- and long-term weather pattern predictions. Meteorologists 

typically use a combination of models and their own experience to 

develop local forecasts and warnings. In addition, numerical weather 

models are a critical source for forecasting weather up to 2 weeks in 

advance and forecasting long-term climate changes. 

Numerical Weather Models 

NWS leases high-performance supercomputers to execute numerical 

calculations supporting weather prediction and climate modeling. In 2002, 

NWS awarded a $227 million contract to lease high-performance 

supercomputers to run its environmental models from 2002 through 

September 2011. Included in this contract are an operational 

supercomputer used to run numerical weather models, an identical 

backup supercomputer located at a different site, and a research and 

development supercomputer on which researchers can test new analyses 

and models. 

Supercomputers 

Weather Forecast Office 

Operations: An Overview 

NWS delivers a broad spectrum of climate, weather, water, and space 

weather services through the coordinated activities of the weather 

forecast offices; river forecast centers; and national centers, which 

comprise the National Centers for Environmental Prediction, aviation

center weather service units, and other support offices around the 

country. Each weather forecast office currently has a fixed geographic 

area for which it provides local warnings and forecasts. These offices are 

Page 9 GAO-07-650 Weather Forecasting
staffed by meteorologists who produce a suite of weather products, 

including short-term and 3- to 5-day extended forecasts as well as aviation

and marine weather products, and monitor for severe weather events. 

Predictive weather services (outlooks, forecasts, watches, warnings, 

advisories, and other supporting information) are produced through a 

process that incorporates observational data, numerical model output and 

statistical guidance, and human expertise. These services are delivered in

many forms, including textual, digital, and graphical formats. Local offices 

and regional and national centers coordinate as needed in the preparation 

of these services. 

When a severe weather event occurs, forecast office managers often ask 

their staffs to work overtime so that there are enough personnel available

to do both the normal forecasting work and the watches and warnings 

required by the severe event. If a weather forecast office is unable to 

provide forecast and warning functions due to an interruption in 

communications or power, an adjacent office will temporarily assume 

those duties. For example, during Hurricane Katrina, several weather 

forecast offices in Louisiana, Mississippi, and Alabama were damaged and 

inoperable. These offices’ responsibilities were transferred to backup sites

in Texas, Florida, Alabama, and Tennessee. During this transfer of 

operations, called service backup, the backup office gains access to the 

original office’s system data and calls in extra personnel to handle the 

workload of both offices.

According to NWS officials, due to software limitations in AWIPS, 

switching to a backup service arrangement can take 60 to 90 minutes, and, 

when invoked, the backup office must assume all products and services 

for the defunct office. In addition, the current backup arrangement can 

cause duplicate work. For example, one office could produce a forecast 

for its own geographic area, and then would need to produce a separate 

forecast for its backup partner. 

Page 10 GAO-07-650 Weather Forecasting
Over the past few years, NWS has evaluated options for an alternate 

concept of operations. According to agency officials, in 2005, NWS’s board 

of executive advisors (called the Corporate Board)2

 noted that a 

constrained budget, high labor costs, difficulty in training and developing

its employees, and a lack of flexibility in how the agency was operating 

were making it more difficult for the agency to continue to perform its 

mission. In August 2005, the board chartered a working group to evaluate

the roles, responsibilities, and functions of weather offices nationwide and 

to make a proposal for a more efficient concept of operations. The group 

was given a set of guiding principles, including that the proposed concept

should (1) be cost-effective, (2) ensure that there would be no degradation 

of service, (3) ensure that weather services nationwide were equitable, and 

(4) not reduce the number of forecast offices nationwide. 

NWS Explored Options for 

a New Concept of 

Operations and Proposed 

One Option for Further 

Study 

The group gathered input from various agency stakeholders and other 

partners within NOAA and considered multiple alternatives. They 

dismissed all but one of the alternative concepts because they were not 

consistent with the guiding principles. In its December 2005 proposal, the 

working group proposed the remaining concept, in which several offices 

with the same type of weather and warning responsibilities, climate, and 

customers would be grouped together to share responsibilities, 

particularly when there is a high-impact weather event affecting one of the 

offices. NWS called this a “clustered peer concept of operations.” 

Under the clustered peer concept of operations, offices would share the 

workload associated with routine services, such as 7-day forecasts. During 

a high-impact weather event—such as a severe storm, flood, or wildfire—

the offices would redistribute the workload to allow the impacted office to 

focus solely on the event, while the other offices in the cluster would pick 

up the impacted office’s routine services. In addition, offices in a cluster 

could occasionally allocate the work so that certain offices would take on

forecasting responsibilities while other offices focused on other tasks, 

such as training, customer outreach, and research. In this manner, peer 

 2

NWS’s Corporate Board is chaired by the Director of the National Weather Service and

composed of senior officials responsible for different aspects of the agency’s mission, 

including the Chief Information Officer and the Directors of the Office of Climate, Water, 

and Weather Services; the National Centers for Environmental Prediction; and the NWS 

regions. The board meets at least twice annually to discuss NWS’s budget and strategic 

issues. It also holds meetings, as needed, to focus on special topics, such as an assessment 

of weather services provided during Hurricane Charley in 2004. 

Page 11 GAO-07-650 Weather Forecasting
offices could help supplement staffing needs, and the workload across 

multiple offices could be more efficiently balanced. 

Some of these benefits were demonstrated in recent years by a pair of

weather forecasting offices. The Detroit and Grand Rapids offices 

established a technology workaround to allow them to more efficiently 

switch to backup operations between the two offices. This technology 

improvement allows them to switch to backup mode instantaneously, 

rather than taking 60 to 90 minutes using current AWIPS technology. To 

date, the two offices have used this backup capability to share workload 

during high-impact weather events; during system upgrades; and while 

conducting officewide training, research, and public outreach. 

NWS Developed a 

Prototype Plan for the 

Alternative Concept of 

Operations 

In February 2006, NWS’s Corporate Board chartered a team to develop a 

prototype plan for the clustered peer concept of operations. Between 

February and November 2006, the team established a preliminary concept 

for what the prototype would entail, conducted a laboratory exercise to 

determine AWIPS’s ability to support the prototype, and—on the basis of 

the findings from the laboratory exercise—developed a prototype plan. 

The team’s preliminary concept was for the prototype to consist of four 

clusters ranging in size from four to eight weather forecast offices. 

However, in September 2006, a laboratory exercise conducted between 

NOAA’s Global Systems Division in Boulder and NWS’s Central Region 

Headquarters in Kansas City showed that AWIPS was not capable of 

supporting clusters of more than four offices. The laboratory results 

showed that with minor modifications, AWIPS was capable of allowing a 

more efficient and timely switch of operations between two weather 

forecast offices for backup purposes, and that, in some cases, functions 

could be shared among four offices. Officials in charge of the laboratory 

exercise estimated that larger clusters would not be possible until the next

version of AWIPS is deployed—a milestone currently planned for 2010. 

The laboratory exercise also identified several problems with system 

performance when weather forecast offices work as a cluster of two or 

more offices. Specifically, each weather forecast office has its own set of 

customized tools supporting each office’s graphical forecast editor. The 

laboratory exercise found that running these tools when sharing functions 

among multiple offices caused AWIPS to slow down or crash. 

On the basis of the findings from the laboratory exercise, the prototype 

team developed a plan to demonstrate opportunities for improved 

efficiency. The goal of the prototype was to validate that a group of 

Page 12 GAO-07-650 Weather Forecasting
weather forecast offices, operating as one unit, could be more efficient 

than each office operating on its own. The plan entailed sharing 

forecasting responsibilities between 2, and subsequently among 4, weather 

forecast offices using a single workstation in each office. The prototype 

was to be a hands-on demonstration conducted at 20 weather forecast 

offices, clustered in 4 different areas of the country: the Pacific Northwest, 

Great Plains, Great Lakes, and Southeast (see fig. 4). These areas were 

selected because they present different forecasting challenges, involve all 

of the regions in the contiguous United States, and engage a variety of field 

offices. In November 2006, the Corporate Board approved moving forward 

with the prototype plan.

Page 13 GAO-07-650 Weather Forecasting
Figure 4: NWS Offices Involved in the Clustered Peer Concept of Operations

Prototype

Sources: NWS (data) and Map Resources (map).

Seattle

Portland

Pacific Northwest Cluster

Great Plains Cluster

Pendleton

Spokane

Amarillo

Lubbock Wichita

Tulsa

Norman

Dodge City

Weather Forecast Office

Green Bay

Milwaukee

Grand Rapids

Detroit

Cleveland

Northern Indiana

Northern Lower Michigan Chicago

Tallahassee

GreenvilleSpartanburg

Jacksonville

Atlanta

Charleston

Wilmington

Columbia

Great Lakes Cluster

Southeast Cluster

Following the Corporate Board’s decision to proceed, NWS initiated the 

first of three phases of its prototype, which it planned to continue over the 

next 2 years. The first phase was expected to take place between 

November 2006 and June 2007 and was to include planning activities. As of 

March 2007, NWS had established a program manager and initiated efforts 

to plan for the prototype, including documenting how the prototype would 

be executed and developing training for employees who would participate 

in the prototype. 

NWS Initiated a 2-Year 

Prototype and Then 

Suspended It 

Page 14 GAO-07-650 Weather Forecasting
During the second phase, which NWS expected to take place between 

August 2007 and February 2008, the agency planned to have 20 weather 

offices share forecasting responsibilities in 2-office pairs. During the third 

phase, which NWS expected to take place between April 2008 and April 

2009, the agency planned to have the same 20 weather offices share 

forecasting responsibilities in 4-office clusters. Each phase was to be 

preceded by both a laboratory exercise to test the new expanded functions

and a board decision on whether to move ahead, based on the results of 

the laboratory exercise. NWS planned to conclude prototype operations 

and issue a report on the prototype by July 2009. By August 2009, NWS 

intended to make a decision on whether to implement the new concept of 

operations on a national basis. NWS’s schedule for the prototype is 

provided in figure 5. A more detailed description of the phases of the 

prototype is provided in appendix II.

Figure 5: Preliminary Schedule for NWS’s Clustered Peer Concept of Operations Prototype, as of November 14, 2006 

Source: GAO analysis of NWS data. 

Working 

group 

chartered

Working 

group 

proposal 

delivered

Lab 1 

testing

2005 2006 2007 2008 2009

Board 

decision

Prototype 

report to be 

delivered

Lab 2 

testing

Phase 1 Phase 2 Phase 3

Lab 3 

testing

Fiscal year

Board 

decision

Board 

decision

Board 

decision

Board 

decision

Prototype 

team

chartered

J FMAMJ J A S OND J FMAMJ J A S OND J FMAMJ J A S OND J FMAMJ J A S OND J FMAMJ J A S OND

These efforts were halted when the prototype was suspended. On March 

23, 2007, Commerce’s Under Secretary for Oceans and Atmosphere 

directed NWS’s director to place all activities associated with NWS’s 

concept of operations on hold pending a review by the Deputy Under 

Secretary. NWS’s director subsequently issued a memorandum noting that

the prototype was being suspended because of concerns about the 

agency’s approach expressed by some Members of Congress and by NOAA 

leadership, the private sector, the agency’s workforce, and the agency’s 

emergency management partners. The director noted that the prototype 

would be put on hold until a reevaluation could be undertaken. 

Page 15 GAO-07-650 Weather Forecasting
Before the prototype was suspended, NWS developed and approved a 

2-year plan for its prototype without conducting an analysis of the relative 

costs and benefits of the initiative. Leading organizations have developed 

best practices for justifying prototyping efforts—including analyzing the 

expected costs and quantified benefits of alternatives before beginning

prototyping activities—and identifying plans, schedules, and risks.3

Furthermore, federal guidance4

 notes that in conducting a cost-benefit 

assessment, tangible and intangible benefits and costs should be 

identified, assessed, and reported. 

NWS Has Not Yet 

Sufficiently Justified 

Its Prototype 

The prototype plan approved by the Corporate Board in November 2006 

identifies the schedules, risks, and expected benefits of the prototype. The 

prototype was estimated to cost $9.3 million, and the Corporate Board’s 

financial investment review committee approved spending $1.5 million on 

the prototype in the first half of fiscal year 2007. As of March 2007, NWS 

had spent approximately $724,000 on the prototype (see table 1). 

Moreover, the prototype plan identifies qualitative benefits of the new 

concept of operations—including increased efficiency and time for other 

activities (such as outreach and research), the ability to switch to a 

backup site more quickly when a weather forecast office is disabled due to

loss of power or communications, and an improved ability to focus on 

high-impact events. 

 3

Carnegie Mellon University Software Engineering Institute, Capability Maturity Model

®

Integration for Development, Version 1.2 (Pittsburgh, PA: August 2006). The Carnegie 

Mellon University Software Engineering Institute, recognized for its expertise in software

and system processes, has developed the Capability Maturity Model®

Integration for

Development to evaluate, improve, and manage system and software development

processes.

4

Office of Management and Budget, Guidelines and Discount Rates for Benefit-Cost 

Analysis of Federal Programs, OMB Circular A-94 (Washington, D.C.: Oct. 29, 1992). 

Page 16 GAO-07-650 Weather Forecasting
Table 1: Clustered Peer Concept of Operations Prototype Costs, as of March 2007 

Fiscal year

Approved

expenditures

Actual

expenditures

Estimated 

expendituresa

Total

2005 $4,125 $4,125 — $4,125

2006 430,879 430,879 — 430,879

2007 1,500,000 288,907 $2,911,093 3,200,000

2008 — 3,000,000 3,000,000

2009 — 2,700,000 2,700,000

Total $1,935,004 $723,911 $8,611,093 $9,335,004

Source: GAO analysis of NWS data. 

a

NWS has not yet approved these estimates. 

However, NWS did not conduct a cost-benefit analysis to justify its plans 

for the prototype or the $1.9 million that has already been approved for the 

prototype. 

By not conducting a cost-benefit analysis before beginning the prototype, 

NWS lacked assurance that its approach was cost-effective. If NWS 

decides to proceed with the prototype without conducting a cost-benefit 

analysis, the agency will have little basis to ensure that its investment is 

sound or that the options it is undertaking provide the greatest return on 

investment. Specifically, a cost-benefit analysis could help ascertain which 

aspects of the prototype promise the most benefits and enable NWS to 

adjust its plans accordingly. For example, a cost-benefit assessment might 

show that more benefits could be gained for the least cost by moving to a 

2-office cluster, rather than a 4-office cluster. 

In its prototype plan, NWS identified goals and selected measures to 

evaluate during its prototype activities, but it did not establish a rigorous 

evaluation plan. Specifically, the agency had not defined a comprehensive 

set of measures, how it planned to compare prototype results with 

baseline performance, or how its measures supported the goals of the 

prototype. 

NWS Lacks a 

Rigorous Evaluation 

Plan for Assessing Its 

Prototype 

According to best practices in leading organizations, well-developed 

evaluation plans increase the likelihood that evaluations will yield sound 

results, thereby supporting effective policy decisions. Key features of a 

well-developed evaluation plan include well-defined, clear, and 

measurable objectives or goals; quantitative measures for determining 

program performance as compared with baseline performance; and a 

Page 17 GAO-07-650 Weather Forecasting
linkage between the measures and the program goals. Such plans are 

essential to ensuring, among other things, that programs are executed 

properly and that funding requests are developed appropriately. In July 

2006, we reported that NWS had not yet established plans, time lines, and 

measures for evaluating its prototype, and we recommended that NWS do

so before beginning the prototype.5

NWS’s prototype plan identifies evaluation goals and measures, and notes 

that NWS plans to contract with an independent entity to perform the 

evaluation. The prototype goals were to (1) validate that a group of 

weather forecast offices, operating as one unit, could be more effective 

and efficient than each office on its own; (2) increase focus during highimpact events through more efficient resource allocation; (3) gain 

efficiencies through the use of resource allocation during benign weather 

that can be applied to training, outreach, and professional development; 

and (4) use some of the efficiencies gained through the prototype to 

further the NOAA mission. Quantitative and qualitative measures that were 

to be used in evaluating the prototype included the number of times that 

work was transferred to another forecast office to focus on high-impact 

events, the number of additional hours made available during benign 

weather for nonforecast work, and customer perceptions of service quality 

and timeliness. The agency stated that system performance and 

hardware/software functionality would have been evaluated using staff 

feedback, performance monitoring software, and usage logs, while 

customer perceptions would have been evaluated using customer 

satisfaction surveys. 

However, NWS did not establish an evaluation plan that identifies a full set 

of applicable measures, what baseline performance its prototype would be

compared with, and how its measures would support the goals of the 

prototype. Specifically, while the agency listed various items that it 

planned to measure in its prototype plan, it did not include quantitative 

measures of product quality, timeliness, the impact on the efficiency of the 

office that is receiving the increased workload, or plans to assess the 

sensitivity of its results to weather-related situations. In addition, the 

agency did not establish the baseline performance against which it would 

compare its prototype, so it was not clear how prototype results would be 

 5

GAO, Weather Forecasting: National Weather Service Is Planning to Improve Service 

and Gain Efficiency, but Impacts of Potential Changes Are Not Yet Known, GAO-06-792 

(Washington, D.C.: July 14, 2006). 

Page 18 GAO-07-650 Weather Forecasting
assessed. Furthermore, the measures that NWS identified did not directly 

support the goals. For example, the number of times functions are 

transferred to another site does not demonstrate increased efficiency. For 

this measure to support the desired goal of increased efficiency, the 

measure would need to be supplemented with an assessment of how the 

office that transferred its workload used the newly available time and how 

the receiving site handled its increased workload. 

If NWS were to proceed with its prototype without a rigorous evaluation 

plan, the agency would run an increased risk that it could not sufficiently 

measure the impact of planned changes on its performance, and would 

subsequently make decisions affecting the nation’s weather service on the 

basis of incomplete and flawed data.

NWS involved internal stakeholders in planning its prototype, including 

having selected stakeholders review plans and provide feedback on work 

products and laboratory activities. However, NWS did not involve external 

stakeholders or establish a plan for stakeholder involvement throughout 

the prototype. Leading organizations routinely involve relevant 

stakeholders when considering operational or process changes.6

 In 

addition, NWS has agreed to negotiate with its employees’ union, the 

National Weather Service Employees Organization, whenever 

organizational changes could affect working conditions.7

 Moreover, best 

practices call for developing a plan for stakeholder involvement that 

includes identifying relevant stakeholders, the roles and responsibilities of 

the relevant stakeholders, and a schedule for stakeholder involvement 

throughout the project. 

NWS Involved 

Internal Stakeholders 

in Planning the 

Prototype, but Did 

Not Involve External 

Stakeholders or 

Establish a Plan for 

Stakeholder 

Involvement 

In planning its prototype, NWS involved internal stakeholders, who were 

identified by the NWS Director of Strategic Plans as forecasters, other field 

and headquarters staff, and union representatives. For example, when 

 6

GAO, Information Technology Investment Management: A Framework for Assessing and

Improving Process Maturity, GAO-04-394G (Washington, D.C.: March 2004); and 

Capability Maturity Model

®

Integration for Development, Version 1.2. 

7

NWS, National Collective Bargaining Agreement Between The National Weather Service 

and The National Weather Service Employees Organization (Oct. 25, 2001). This 

agreement calls for NWS to negotiate with the union on the impact and implementation of 

changes affecting working conditions before they can be implemented. The agreement also 

allows the agency to avoid the need for negotiations if the union has sufficient 

predecisional involvement. 

Page 19 GAO-07-650 Weather Forecasting
deciding which offices to group into clusters, meteorologists-in-charge 

were asked for their input on cluster criteria and preferences.

Furthermore, one meteorologist from each of eight potential prototype 

weather forecast offices participated in the laboratory test. Their feedback

on the new concept of operations was acquired through questionnaires, 

electronic logs, and debriefings at the end of each week. In addition, the 

employees’ union provided input before the decision was made to begin 

the prototype. Both the initial working group and the prototype team 

included union representatives. Also, the union president participated in

Corporate Board meetings that included discussions of the clustered peer

prototype, and a union representative observed the final week of the 

laboratory test. 

NWS also intended to work with external stakeholders (including 

emergency managers and other public safety officials, the media and other 

users of NWS information services, and the general public) after the 

prototype was complete to assess the results. Specifically, NWS planned to

identify customer perceptions of service as one basis on which to evaluate

the prototype. Furthermore, in an effort to make its plans available to 

employees and customers, NWS developed a public Web site that contains

the plans and reports for the prototype.8

However, NWS did not request input from external stakeholders, including

emergency managers, during the past 20 months that it was planning and 

initiating the prototype. NWS officials reported that they did not involve 

external stakeholders because changes in the concept of operations during 

the prototype should not be noticeable to outside stakeholders; that is, 

external stakeholders should not notice any difference in service. 

Although this goal has merit, external stakeholders need to be more 

involved in the prototype for it to gain acceptance and succeed. 

Specifically, external stakeholders need to understand what is being 

tested, have input into the tests, be alert for differences under the 

prototype, and have formal mechanisms for sharing their observations. In 

addition, NWS did not have a comprehensive plan for involving 

stakeholders in its prototyping activities. Specifically, it did not identify 

key external stakeholders or stakeholder representatives, determine the 

responsibilities of internal or external stakeholders, or develop a time line 

for involving stakeholders and addressing their comments. 

 8

NWS, Focus on the Future (Silver Spring, MD: Apr. 5, 2007),

http://www.weather.gov/com/digitalera/ (viewed on Apr. 10, 2007). 

Page 20 GAO-07-650 Weather Forecasting
If NWS were to proceed with its prototype without a plan for involving 

internal and external stakeholders, it would not be able to ensure that 

employee and customer interests are anticipated, identified, and 

addressed. 

NWS stated that it would ensure that there would be no degradation of 

service during or after its prototype, and that it had plans in place to 

mitigate the risk of any service degradation. Specifically, the agency

conducted and planned to continue to conduct laboratory exercises to 

assess the impact of the prototype before moving into each phase. In 

addition, NWS planned to run the prototype software on a single, isolated 

workstation in each office, while maintaining the existing AWIPS software 

on all other workstations. Established procedures would allow offices to 

return to the baseline software and operational practices should problems 

have developed. Furthermore, NWS planned to monitor the performance 

of all activities and the quality and timeliness of products. 

Looking forward, an NWS official stated that they also would ensure that 

there would be no degradation of service if the Corporate Board decided 

to implement this alternative concept of operations on a national basis 

after the prototype was completed. Agency officials explained that the 

agency would use its standard procedures for testing and validating 

systems and software to ensure that there would be no degradation of 

service before moving them into operations. 

Although there is merit in NWS’s goal of improving its operational 

efficiency and allowing faster backup service during severe weather, the

agency lacked the rigorous planning it needed to justify, evaluate, and 

obtain stakeholder involvement in its planned—and now suspended—

prototype of a new concept of operations. Specifically, the agency 

developed a prototype plan that identified activities, gates, and decisions, 

but it did not perform a cost-benefit assessment to justify the prototype. 

Also, NWS identified goals and selected measures for evaluating its 

prototype, but it did not establish an evaluation plan that clearly identifies

a comprehensive set of measures or how these measures would be 

compared with baseline performance. Furthermore, the agency involved

internal stakeholders in planning for its prototype, but it did not seek input 

from external stakeholders or establish a plan for stakeholder involvement 

throughout the prototype. These matters will be important to consider 

should NWS decide to proceed with the prototype. Without undertaking a 

more rigorous approach, NWS would run an increased risk that its 

NWS Planned to 

Ensure That the 

Prototype Would Not 

Degrade Its Service 

Conclusions 

Page 21 GAO-07-650 Weather Forecasting
prototype will not be a sound investment, and that any resulting decisions 

could be based on flawed data and analysis. Taking this more rigorous 

approach to the prototype should not preclude NWS from considering 

options for improving the ability to switch to a backup site when a 

weather forecast office becomes disabled due to loss of power or 

communications. 

We are making four recommendations to the Secretary of Commerce. If 

NWS decides to proceed with its prototype, we recommend the Secretary

direct the Assistant Administrator for the National Weather Service to 

• develop a cost-benefit analysis for the clustered peer approach before 

implementing the prototype in any weather forecasting office; 

• develop an evaluation plan that includes a comprehensive set of measures 

that are linked to prototype goals and identifies the baseline performance 

that the prototype will be compared with before implementing the 

prototype in any weather forecasting office; and

• develop a plan for internal and external stakeholder involvement, which 

includes a list of relevant stakeholders, roles and responsibilities for these 

stakeholders, and a schedule for when stakeholders should be involved 

before implementing the prototype in any weather forecasting office. 

In addition, we recommend that the Secretary direct the Assistant 

Administrator for the National Weather Service to evaluate moving 

forward with technology upgrades to the AWIPS processing system to 

allow weather forecast offices to switch to backup service more quickly

during high-impact weather situations; these upgrades could be handled as

an initiative that is separate and distinct from broader changes in the 

agency’s concept of operations. 

We received written comments on a draft of this report from Commerce 

(see app. III). In the department’s response, the Deputy Secretary of 

Commerce agreed with our recommendations to conduct a thorough costbenefit analysis, develop a rigorous evaluation plan with fully defined

performance measures, and involve internal and external stakeholders. 

The response stated that our findings were included in the Under 

Secretary’s recent review of the concept of operations prototype. On the 

basis of that review, the NOAA Administrator directed NWS to cease all 

activities associated with the concept of operations prototype, and to 

Recommendations for 

Executive Action 

Agency Comments 

and Our Evaluation 

Page 22 GAO-07-650 Weather Forecasting
undertake a comprehensive and analytical review to determine new and 

revised requirements for NWS products and services. This review is 

expected to determine what, if any, changes are required. Should NWS 

decide to proceed with a new concept of operations, we believe that our 

recommendations to justify, evaluate, and involve stakeholders in any 

such changes would still be relevant and useful. 

In addition, Commerce agreed with our recommendation to evaluate

moving forward with technology upgrades to the AWIPS processing 

system to allow weather forecast offices to switch to backup service more 

quickly during high-impact weather situations. The department stated that

NWS has established a proposal for improving its service backup 

capability, expects to approve requirements for the backup capability by 

September 2007, and plans to deploy this capability by September 2008. 

As we agreed with your offices, unless you publicly announce the contents 

of this report earlier, we plan no further distribution of it until 30 days 

from the report date. At that time, we will send copies of this report to 

interested congressional committees, the Secretary of Commerce, the

Director of the Office of Management and Budget, and other interested 

parties. In addition, this report will be available at no charge on our Web 

site at www.gao.gov. 

If you have any questions about this report, please contact me at 

(202) 512-9286 or by e-mail at pownerd@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on the 

last page of this report. GAO staff who made major contributions to this 

report are listed in appendix IV. 

David A. Powner 

Director, Information Technology 

 Management Issues 

Page 23 GAO-07-650 Weather Forecasting
Appendix I: Objectives, Scope, and Appendix I: Objectives, Scope, and Methodology 

Methodology 

Our objectives were to (1) determine the current status of and plans for 

the prototype, (2) evaluate whether the justification for the prototype is 

sufficient, (3) determine whether the National Weather Service’s (NWS) 

plans to evaluate the prototype are adequate, (4) evaluate whether NWS is

sufficiently involving stakeholders in its prototype plans, and

(5) determine how NWS plans to ensure that there will be no degradation 

of service during and after the prototype. 

To determine the current status of and plans for the prototype, we 

reviewed agency documentation on the planned prototype’s goals, major 

components, and milestones, such as the Corporate Board’s meeting 

minutes, the prototype plan, and the preprototype laboratory report. We 

also interviewed NWS officials to obtain clarifications on agency plans. 

To evaluate whether the justification for the prototype is sufficient, we 

reviewed program documents such as the prototype plan and other NWS 

planning documents, presentations, and communications about the goals 

and costs of the prototype. We compared these planning documents with 

best practices for justifying process-improvement proposals by the 

Carnegie Mellon University Software Engineering Institute’s Capability 

Maturity Model®

Integration for Development1 and federal guidance on

conducting cost-benefit analyses.2

 We also interviewed agency officials 

and employees who were involved in the planning of the prototype and

laboratory exercise to determine the potential costs, benefits, and risks of 

the prototype. 

To determine whether NWS plans to evaluate the prototype are adequate,

we reviewed planning documents, such as the prototype plan and NWS 

customer satisfaction scores, and compared them with best practices for 

 1

Carnegie Mellon University Software Engineering Institute, Capability Maturity Model

®

Integration for Development, Version 1.2 (Pittsburgh, PA: August 2006). Capability 

Maturity Model®

 and Capability Maturity Modeling are registered in the U.S. Patent and 

Trademark Office. CMM is a service mark of Carnegie Mellon University. 

2

Office of Management and Budget, Guidelines and Discount Rates for Benefit-Cost 

Analysis of Federal Programs, OMB Circular A-94 (Washington, D.C.: Oct. 29, 1992). 

Page 24 GAO-07-650 Weather Forecasting
Appendix I: Objectives, Scope, and 

Methodology 

evaluating prototypes from the Capability Maturity Model®Integration

for Development and GAO’s guidance for developing and assessing 

evaluation plans.3

 In addition, we interviewed NWS officials who were 

responsible for planning the prototype activities. 

To evaluate whether NWS is sufficiently involving stakeholders in its 

prototype plans, we reviewed NWS’s plans, presentations, and other 

communications about the prototype, including an external Web site that 

presented these documents. We compared NWS’s plans for involving 

stakeholders with best practices from the Capability Maturity Model®

Integration for Development. We spoke with members of stakeholder 

groups to determine their knowledge of the prototype and their interaction 

with NWS. In addition, we interviewed NWS officials and the president of 

the National Weather Service Employees Organization to determine the 

level of involvement of the employee union in planning for the prototype.

To determine how NWS plans to ensure that there will be no degradation 

of service during and after the prototype, we reviewed planning 

documents—including the prototype plan, presentations, and 

communications regarding the prototype—and interviewed NWS officials. 

We performed our work at NWS headquarters in the Washington, D.C., 

metropolitan area; at NWS offices in Tampa, Florida, Detroit, Michigan, 

and Kansas City, Missouri; and at a National Oceanic and Atmospheric 

Administration (NOAA) office in Boulder, Colorado. We visited the Tampa 

weather forecast office because the president of the employee union and

the vice president of the International Association of Emergency Managers 

are located in the Tampa area. The Detroit weather forecast office was 

included because of its prior experiences in working in a paired backup 

with expanded domains. We visited the NWS Central Region Headquarters

in Kansas City because of its participation in the first laboratory exercise 

and because it is where the prototype team chair worked. The NOAA 

 3

GAO, Equal Employment Opportunity: DOD’s EEO Pilot Program Under Way, but 

Improvements Needed to DOD’s Evaluation Plan, GAO-06-538 (Washington, D.C.: May 5,

2006); and Earned Income Tax Credit: Implementation of Three New Tests Proceeded

Smoothly, But Tests and Evaluation Plans Were Not Fully Documented, GAO-05-92 

(Washington, D.C.: Dec. 30, 2004). 

Page 25 GAO-07-650 Weather Forecasting
Appendix I: Objectives, Scope, and 

Methodology 

Global Systems Division office in Boulder was chosen because of its 

participation in the first laboratory exercise. We performed our work from

October 2006 to April 2007 in accordance with generally accepted 

government auditing standards. 

Page 26 GAO-07-650 Weather Forecasting
Appendix II: NWS Prototype Plans, as of Appendix II: NWS Prototype Plans, as of March 2007

March 2007 

Prior to the suspension of the prototype by the Director of the National 

Weather Service, the prototype team developed a plan for a three-phased 

prototype of the clustered peer concept of operations. As described in the 

plan, phase 1 would include planning activities, such as training; phase 2 

was meant to establish clustered peer operations in 10 pairs of weather 

forecast offices across the country; and phase 3 would include offices 

grouped into 4-office clusters. NWS planned to conclude prototype 

operations and to issue a report on the prototype by July 2009. NWS was 

to decide at that time if it would proceed to implement the clustered peer 

concept of operations on a national basis. Details about each of the 

planned prototype phases are included in the following text. 

Phase 1, the planning phase, began in November 2006 with the Corporate 

Board’s approval of the prototype plan. NWS established a program 

manager to oversee the prototype. In addition, cluster management teams1

representing each cluster were to meet to decide on the rules of 

engagement, common tools, and common forecast methodologies. At the 

end of this phase, a second laboratory exercise was planned to validate 

that software upgrades and hardware additions to support the prototype 

were operationally ready. The second laboratory exercise was scheduled 

for June 2007 and was to involve only weather forecast offices that are 

currently service-backup pairs within the original four geographic cluster 

areas. Upon completion of this exercise, the Corporate Board was to 

decide on whether to initiate Phase 2 of the prototype. 

Phase 2 was planned to begin in August 2007. During this phase, NWS 

planned to conduct training, develop guidance for how and when offices 

would take over activities for other offices (called playbooks), define the 

cluster methodology, and have 10 pairs of offices operate as clustered 

peers. Also during this phase, NWS planned to develop the software 

upgrades needed for expanded operations in the next phase. Near the end 

of this phase, in February 2008, NWS planned to conduct a third laboratory 

exercise to test the software necessary for moving to clusters of 4 offices. 

After completion of this exercise, the Corporate Board was to decide on 

whether to initiate Phase 3 of the prototype. 

 1

The cluster management team provides operational oversight to the cluster. The team is 

composed of the meteorologists-in-charge from the cluster weather forecast offices, 

hydrologists-in-charge from the river forecast centers, and a regional representative. 

Page 27 GAO-07-650 Weather Forecasting
Appendix II: NWS Prototype Plans, as of 

March 2007

Phase 3 was planned to begin in April 2008 and was to determine if 

clusters of 4 offices can work as a multioffice team sharing forecasting 

responsibilities. NWS planned to complete a preliminary prototype report

in July 2009 and present it to the Corporate Board. At that time, the board

would have decided whether to implement the clustered peer concept of 

operations across the country. 

Page 28 GAO-07-650 Weather Forecasting
Appendix III: Comments from the Department Appendix III: Comments from the of Commerce 

Department of Commerce 

Page 29 GAO-07-650 Weather Forecasting
Appendix III: Comments from the Department 

of Commerce 

Page 30 GAO-07-650 Weather Forecasting
Appendix III: Comments from the Department 

of Commerce 

Page 31 GAO-07-650 Weather Forecasting
Appendix IV: 

St

GAO Contact and 

aff Acknowledgments

Page 32 GAO-07-650 

Appendix IV: GAO Contact and Staff 

Acknowledgments 

David A. Powner, (202) 512-9286 or pownerd@gao.gov

In addition to the contact person named above, Colleen Phillips, Assistant 

Director; Kate Agatone; Neil Doherty; Amos Tevelow; and Jessica 

Waselkow made key contributions to this report. 

Weather Forecasting

GAO Contact 

Staff 

Acknowledgments 

(310824) 
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER