Weather Forecasting: National Weather Service Is Planning to Improve Service and Gain Efficiency, but Impacts of Potential Changes Are Not Yet Known
Published: 2006-07-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: To provide accurate and timely weather forecasts, the National Weather Service (NWS) uses systems, technologies, and manual processes to collect, process, and disseminate weather data to its nationwide network of field offices and centers. After completing a major modernization program in the 1990s, NWS is seeking to upgrade its systems with the goal of improving its forecasting abilities, and it is considering changing how its nationwide office structure operates in order to enhance efficiency. GAO was asked to (1) evaluate NWS's efforts to achieve improvements in the delivery of its services through system and technology upgrades, (2) assess agency plans to achieve service improvements through training its employees, and (3) evaluate agency plans to revise its nationwide office configuration and the implications of these plans on local forecasting services, staffing, and budgets.
section_title: What GAO Found
paragraphs: NWS is positioning itself to provide better service through over $315 million in planned upgrades to its systems and technologies. In annual plans, the agency links expected improvements in its service performance measures with the technologies and systems expected to improve them. For example, NWS expects to reduce the average error in its forecasts of hurricane paths by approximately 20 nautical miles between 2005 and 2011 through a combination of upgrades to observation systems, better hurricane forecast models, enhancements to the computer infrastructure, and research that will be transferred to forecast operations. Also, NWS expects to increase tornado warning lead times from 13 to 15 minutes by the end of fiscal year 2008 after the agency completes an upgrade to its radar system and realizes benefits from software improvements to its forecaster workstations. NWS also provides training courses for its employees to help improve its forecasting services, but the agency's process for selecting training lacks sufficient oversight. Program officials propose and justify training needs on the basis of up to eight different criteria--including whether a course is expected to improve NWS forecasting performance measures, support customer outreach, or increase scientific awareness. Many of these course justifications appropriately demonstrate support for improved forecasting performance. For example, training on how to more effectively use forecaster workstations is expected to help improve tornado and hurricane warnings. However, in justifying training courses, program officials routinely link courses to NWS forecasting performance measures. For example, in 2006, almost all training needs were linked to expectations for improved performance--including training on cardiopulmonary resuscitation, spill prevention, and systems security. The training selection process did not validate or question how these courses could help improve weather forecasts. Overuse of this justification undermines the distinctions among different training courses and the credibility of the course selection process. Additionally, because the training selection process does not clearly distinguish among courses, it is difficult to determine whether sufficient funds are dedicated to the courses that are expected to improve performance. To improve its efficiency, NWS plans to develop a prototype of a new concept of operations, an effort that could affect its national office configuration, including the location and functions of its offices nationwide. However, many details about the impact of any proposed changes on NWS forecast services, staffing, and budget have yet to be determined. Further, the agency has not yet determined key activities, timelines, or measures for evaluating the prototype of the new office operational structure. As a result, it is not evident that NWS will collect the information it needs on the impact and benefits of any office restructuring in order to make sound and cost-effective decisions.

section_title: Letter
paragraphs: The National Weather Service’s (NWS) ability to forecast the weather  affects the life and property of every American. The agency’s basic mission  is to provide storm and flood warnings and weather forecasts for the  United States, its territories, and adjacent oceans and waters—in order to  protect life and property and to enhance the national economy. NWS  operations also support other agencies by providing aviation and marine- related weather forecasts and warnings.
To carry out its mission, NWS uses a variety of systems, technologies, and  manual processes to collect, process, and disseminate weather data to and  among its network of field offices and regional and national centers. In the  1980s and 1990s, NWS undertook a nationwide modernization program to  upgrade its systems and technologies and to consolidate its field office  structure. Today, with the modernization completed, NWS continues to  seek ways to upgrade its systems with a goal of further improving its  forecasting abilities. The agency is also considering changing how its  nationwide office structure works in order to enhance its efficiency.
Because of your interest in these plans for continued improvements, you  asked us to (1) evaluate NWS’s efforts to achieve improvements in the  delivery of its services through the upgrades to its systems, models, and  computational abilities; (2) assess agency plans to achieve improvements  in the delivery of its services through the training and professional  development of its employees; and (3) evaluate agency plans to revise its  nationwide office configuration and the implications of these plans on  local forecasting services, staffing, and budgets.
To address these objectives, we reviewed NWS plans for system  enhancements, technology improvements, and professional training, and  assessed the extent to which these plans were tied to the agency’s service  improvement goals. We also interviewed officials from NWS and the  National Oceanic and Atmospheric Administration to obtain clarification  on agency plans and goals. To determine the status and potential impact of  any plans to revise the national office configuration, we assessed NWS  reports on ways to enhance its operations and interviewed key officials  involved in these reports.
We conducted our work at NWS headquarters in the Washington, D.C.,  metropolitan area and at NWS offices in Denver, Tampa, and Miami. We  performed our work from October 2005 to June 2006, in accordance with  generally accepted government auditing standards. Additional details on  our objectives, scope, and methodology are provided in appendix I.
subsections: section_title: Background
paragraphs: The mission of NWS—an agency within the Department of Commerce’s  National Oceanic and Atmospheric Administration (NOAA)—is to provide  weather, water, and climate forecasts and warnings for the United States,  its territories, and its adjacent waters and oceans, in order to protect life  and property and to enhance the national economy. NWS is the official  source of aviation- and marine-related weather forecasts and warnings, as  well as warnings about life-threatening weather situations.
In the 1980s and 1990s, NWS undertook a nationwide modernization  program to develop new systems and technologies and to consolidate its  field office structure. The goals of the modernization program were to  achieve more uniform weather services across the nation, improve  forecasts, provide more reliable detection and prediction of severe  weather and flooding, permit more cost-effective operations, and achieve  higher productivity. The weather observing systems (including radars,  satellites, and ground-based sensors) and data processing systems that  currently support NWS operations were developed and deployed under the  modernization program. During this period, NWS consolidated over 250  large and small weather service offices into the office structure currently  in use.
subsections: section_title: NWS Office Structure: An Overview
paragraphs: The coordinated activities of weather facilities throughout the United  States allow NWS to deliver a broad spectrum of climate, weather, water,  and space weather services. These facilities include weather forecast  offices, river forecast centers, national centers, and aviation center  weather service units. The functions of these facilities are described  below.    122 weather forecast offices are responsible for providing a wide  variety of weather, water, and climate services for their local county  warning areas, including advisories, warnings, and forecasts (see fig. 1 for  the current location of weather forecast offices).   13 river forecast centers provide river, stream, and reservoir  information to a wide variety of government and commercial users as well  as to local weather forecast offices for use in flood forecasts and  warnings.   9 national centers constitute the National Centers for Environmental  Prediction, which provide nationwide computer model output and manual  forecast information to all NWS field offices and to a wide variety of  government and commercial users. These centers include the  Environmental Modeling Center, Storm Prediction Center, Tropical  Prediction Center, Climate Prediction Center, Aviation Weather Center,  and Space Environment Center, among others.   21 aviation center weather service units, which are co-located with  key Federal Aviation Administration (FAA) air traffic control centers  across the nation, provide meteorological support to air traffic controllers.
section_title: NWS Relies on Key Systems and Technologies to Fulfill Its Mission
paragraphs: To fulfill its mission, NWS relies on a national infrastructure of systems  and technologies to gather and process data from the land, sea, and air.  NWS collects data from many sources, including ground-based Automated  Surface Observing Systems (ASOS), Next Generation Weather Radars  (NEXRAD), and operational environmental satellites. These data are  integrated by advanced data processing workstations—called Advanced  Weather Interactive Processing Systems (AWIPS)—used by meteorologists  to issue local forecasts and warnings. The data are also fed into  sophisticated computer models running on high-speed supercomputers,  which are then used to help develop forecasts and warnings. Figure 2  depicts the integration of the various systems and technologies and is  followed by a description of each.
section_title: Next Generation Weather Radar (NEXRAD)
paragraphs: NEXRAD is a Doppler radar system that detects, tracks, and determines  the intensity of storms and other areas of precipitation, determines wind  velocities in and around detected storm events, and generates data and  imagery to help forecasters distinguish hazards such as severe  thunderstorms and tornadoes. It also provides information about heavy  precipitation that leads to warnings about flash floods and heavy snow.  The NEXRAD network provides data to other government and commercial  users and to the general public via the Internet.
The NEXRAD network is made up of 158 operational radars and 8  nonoperational radars that are used for training and testing. Of these, NWS  operates 120 radars, the Air Force operates 26 radars, and the FAA  operates 12 radars. These radars are located throughout the continental  United States and in 17 locations outside the continental United States.  Figure 3 shows a NEXRAD radar tower.
section_title: Automated Surface Observing System (ASOS)
paragraphs: ASOS is a system of sensors, computers, display units, and  communications equipment that automates the ground-based observation  and dissemination of weather information nationwide. This system  collects data on temperature and dew point, visibility, wind direction and  speed, pressure, cloud height and amount, and types and amounts of  precipitation. ASOS supports weather forecast activities and aviation  operations, as well as the needs of research communities that study  weather, water, and climate. Figure 4 is a picture of the system, while  figure 5 depicts a configuration of ASOS sensors and describes their  functions.
There are currently 1,002 ASOS units deployed across the United States,  with NWS, FAA, and the Department of Defense (DOD) operating 313, 571,  and 118 units, respectively.
Although NWS does not own or operate satellites, geostationary and polar- orbiting environmental satellite programs are key sources of data for its  operations. NOAA manages the Geostationary Operational Environmental  Satellite (GOES) system and the Polar-orbiting Operational Environmental  Satellite (POES) system. In addition, DOD operates a different polar  satellite program called the Defense Meteorological Satellite Program  (DMSP). These satellite systems continuously collect environmental data  about the Earth’s atmosphere, surface, cloud cover, and electromagnetic  environment. These data are used by meteorologists to develop weather  forecasts and other services, and are critical to the early and reliable  prediction of severe storms, such as tornadoes and hurricanes.
Geostationary satellites orbit above the Earth’s surface at the same speed  as the Earth rotates, so that each satellite remains over the same location  on Earth. NOAA operates GOES as a two-satellite system that is primarily  focused on the United States (see fig. 6). To provide continuous satellite  coverage, NOAA acquires several satellites at a time as part of a series and  launches new satellites every few years. Three satellites, GOES-10, GOES- 11, and GOES-12, are currently in orbit. Both GOES-10 and GOES-12 are  operational satellites, while GOES-11 is in an on-orbit storage mode. It is a  backup for the other two satellites should they experience any  degradation in service. The first in the next series of satellites, GOES-13,  was launched in May 2006, and the others in the series, GOES-O and  GOES-P, are planned for launch over the next few years. In addition,  NOAA is planning a future generation of satellites, known as the GOES-R  series, which are planned for launch beginning in 2014.
Unlike the GOES satellites, which maintain a fixed position above the  earth, polar satellites constantly circle the Earth in an almost north-south  orbit, providing global coverage of conditions that affect the weather and  climate. Each satellite makes about 14 orbits a day. As the Earth rotates  beneath it, each satellite views the entire Earth’s surface twice a day.  Currently, there are four operational polar-orbiting satellites—two are  POES satellites and two are DMSP satellites. These satellites are  positioned so that they can observe the Earth in early morning, morning,  and afternoon polar orbits. Together, they ensure that for any region of the  Earth, the data are generally no more than 6 hours old. Figure 7 illustrates  the current configuration of operational polar satellites.
NOAA and DOD plan to continue to launch remaining satellites in the  POES and DMSP programs, with final launches scheduled for 2007 and  2011, respectively. In addition, NOAA, DOD, and the National Aeronautics  and Space Administration are planning to replace the POES and DMSP  systems with a state-of-the-art environment monitoring satellite system  called the National Polar-orbiting Operational Environmental Satellite  System (NPOESS). In recent years, we reported on a variety of issues  affecting this major system acquisition.
subsections: section_title: Advanced Weather Interactive Processing System (AWIPS)
paragraphs: AWIPS is a computer system that integrates and displays all  hydrometeorological data at NWS field offices. This system integrates data  from NEXRAD, ASOS, GOES, and other sources to produce rich graphical  displays to aid forecaster analysis and decision making. AWIPS is used to  disseminate weather information to the national centers, weather offices,  the media, and other federal, state, and local government agencies. NWS  deployed hardware and software for this system to weather forecast  offices, river forecast centers, and national centers throughout the United  States between 1996 and 1999. As a software-intensive system, AWIPS  regularly receives software upgrades called “builds.” The most recent  build, called Operational Build 6, is currently being deployed. NWS  officials estimated that the nationwide deployment of this build should be  completed by July 2006. Figure 8 shows a standard AWIPS workstation.
Numerical models are advanced software programs that assimilate data  from satellites and ground-based observing systems and provide short- and  long-term weather pattern predictions. Meteorologists typically use a  combination of models and their own experience to develop local  forecasts and warnings. Numerical weather models are also a critical  source for forecasting weather up to 7 days in advance and forecasting  long-term climate changes. One of NWS’s National Centers for  Environmental Prediction, the Environmental Modeling Center, is the  primary developer of these models within NWS and is responsible for  making new and improved models available to regional forecasters via the  AWIPS system. Figure 9 depicts model output as shown on an AWIPS  workstation.
NWS leases high-performance supercomputers to execute numerical  calculations supporting weather prediction and climate modeling. In 2002,  NWS awarded a $227 million contract to lease high-performance  supercomputers to run its environmental models from 2002 through  September 2011. Included in this contract are an operational  supercomputer used to run numerical weather models, an identical  backup supercomputer located at a different site, and a research and  development supercomputer on which researchers can test out new  analyses and models. The supercomputer lease contract allows NWS to  exercise options to upgrade the processing capabilities of the operational  supercomputer.
section_title: Previous Reports Focused on NWS Modernization Systems Risks
paragraphs: During the 1990s, we issued a series of reports on NWS modernization  systems and made recommendations to improve them. For example, early  in the AWIPS acquisition, we reported that the respective roles and  responsibilities of the contractor and government were not clear and that  a structured system development environment had not been established.  We made recommendations to correct these shortfalls before the system  design was approved. We also reported that the ASOS system was not  meeting specifications or user needs, and recommended that NWS define  and prioritize system corrections and enhancements. On NEXRAD, we  reported that selected units were falling short of availability requirements  and recommended that NWS analyze and monitor system availability on a  site-specific basis and correct any shortfalls. Because of such concerns,  we identified NWS modernization as a high-risk information technology  initiative in 1995, 1997, and 1999.
NWS took a number of actions to address our recommendations and to  resolve system risks. For example, NWS enhanced its AWIPS system  development processes, prioritized its ASOS enhancements, and improved  the availability of its NEXRAD systems. In 2001, because of NWS’s  progress in addressing key concerns and in deploying and using the  AWIPS system—the final component of its modernization program—we  removed the modernization from our high-risk list.
section_title: NWS Established Performance Goals and Tracks Progress against These Goals
paragraphs: In accordance with federal legislation requiring federal managers to focus  more directly on program results, NWS established short- and long-term  performance goals and regularly tracks its actual performance in meeting  these goals. Specifically, NWS established 14 different performance  measures—such as lead time for flash floods and false-alarm rates for  tornado warnings. It also established 5-year goals for improving its  performance in each of the 14 performance measures through 2011. For  example, the agency plans to increase its lead time on tornado warnings  from 13 minutes in 2005 to 15 minutes in 2011. Table 1 identifies NWS’s 14  performance measures, selected goals, and performance against those  goals, when available. Appendix II provides additional information on  NWS’s performance goals.
NWS periodically adjusts its performance goals as its assumptions change.  After reviewing actual results from previous fiscal years and its  assumptions about the future, in January 2006, NWS adjusted eight of its 5- year performance goals to make more realistic predictions for  performance for the next several years. Specifically, NWS made six  performance goals less stringent and two goals more stringent. The six  goals that were made less stringent—and the reasons for the changes—are  the following:    Tornado warning lead time: NWS changed its 2011 goal from 17 to 15  minutes of warning because of delays in deploying new technologies on  NEXRAD radars and a lack of access to FAA radar data.
Tornado warning false-alarm rate: NWS changed its 2011 goal from a  70 to 74 percent false-alarm rate for the same reasons listed above.
Flash flood warning accuracy: NWS changed its 2011 goal from 91 to 90  percent accuracy after delays on two different systems in 2004, 2005, and  2006.
Marine wind speed accuracy: NWS changed its 2011 goal from 67 to 59  percent accuracy after experiencing the delay of marine models and  datasets, a deficiency of shallow water wave guidance, and a reduction in  funds for training.
Marine wave height accuracy: NWS changed its 2011 goal from 77 to 69  percent accuracy for the same reasons identified above for marine wind  speed accuracy.
Aviation instrument flight rule ceiling/visibility: NWS changed its  goal from 48 to 47 percent accuracy in 2006 because of a system delay and  a reduction in funds for training. Goals for 2007 through 2011 remained the  same.
Additionally, the following two goals were made more stringent:    Aviation instrument flight rule ceiling/visibility false-alarm rate:  NWS reduced its expected false-alarm rate from 68 percent to 65 percent  for 2006 because of better than anticipated results from the AWIPS  aviation forecast preparation system and an aviation learning training  course. Goals for the remaining years in the 5-year plan, 2007 to 2011,  remained the same.
Hurricane track forecasts: NWS changed its 2011 hurricane track  forecast goal from 123 to 106 nautical miles after trends in observed data  from 1987 to 2004 showed that this measure was improving more quickly  than expected.
section_title: NWS Is Positioning Itself to Provide Better Service through Upgrades to Its Systems and Technologies
paragraphs: NWS is positioning itself to provide better service through system and  technology upgrades. Over the next few years, the agency plans to upgrade  and improve its systems, predictive weather models, and computational  abilities, and it appropriately links these upgrades to its performance  goals. For example, planned improvements in NEXRAD technology are  expected to help improve the lead times for tornado warnings, while  AWIPS software enhancements are expected to help improve the accuracy  of marine weather forecasts. The agency anticipates continued steady  improvement in its forecast accuracy as it obtains better observation data,  as computational resources are increased, and as scientists are better able  to implement advanced modeling and data assimilation techniques.
subsections: section_title: NWS Has Plans for Upgrading Its Systems, Models, and Computational Abilities
paragraphs: Over the next few years, NWS has plans to spend over $315 million to  upgrade its systems, models, and computational abilities. Some planned  upgrades are to maintain the weather system infrastructure (either to  replace obsolete and difficult-to-maintain parts or to refresh aging  hardware and workstations), while others are to take advantage of new  technologies. Often, the infrastructure upgrades allow NWS to take  advantage of newer technologies. For example, the replacement of an  aging and proprietary NEXRAD subsystem is expected to allow the agency  to implement enhancements in image resolution. Key planned upgrades  for each of NWS’s major systems and technologies are listed below.
NWS has initiated two major NEXRAD improvements. It is currently  replacing an outdated subsystem—the radar data acquisition subsystem— with current hardware that is compliant with open system standards. This  new hardware is expected to enable important software upgrades. In  addition, NWS plans to add a new technology called dual polarization to  this subsystem, which will provide more accurate rainfall estimates and  differentiate various forms of precipitation. Table 2 shows the details of  these two projects.
NWS has seven ongoing and planned improvements for its ASOS system  (see table 3). Many of these improvements are to replace aging parts and  are expected to make the system more reliable and maintainable. Key  subsystem replacements—including the all-weather precipitation  accumulation gauge—are also expected to result in more accurate  measurements.
Selected AWIPS system components have become obsolete, and NWS is  replacing these components. In 2001, NWS began to migrate the existing  Unix-based systems to a Linux system to reduce its dependence on any  particular hardware platform. NWS expects this project, combined with  upgraded information technology, to delay the need for a major  information technology replacement. Table 4 shows planned  improvements for the AWIPS system.
NWS plans to continue to improve its modeling capabilities by (1) better  assimilating data from improved observation systems such as ASOS,  NEXRAD, and environmental satellites; (2) developing and implementing  an advanced global forecasting model (called the Weather Research and  Forecast model) to allow forecasters to look at a larger domain area;  (3) implementing a hurricane weather research forecast model; and  (4) improving ensemble modeling, which involves running a single model  multiple times with slight variations on a variable to get a probability that  a given forecast is likely to occur. NWS expects to spend approximately  $12.7 million in fiscal year 2006 to improve its weather and real-time ocean  models.
subsections: section_title: Supercomputers
paragraphs: NWS is planning to exercise an option within its existing supercomputer  lease to upgrade its computing capabilities to allow more advanced  numerical weather and climate prediction modeling.
section_title: NWS Appropriately Links Its System and Technical Upgrades to Expected Service Improvements
paragraphs: In accordance with federal legislation and policy, NWS’s planned upgrades  to its systems and technologies are expected to result in improved service.  The Government Performance and Results Act calls for federal managers  to develop strategic performance goals and to focus program activities on  obtaining results. Also, the Office of Management and Budget (OMB)  requires agencies to justify major investments by showing how they  support performance goals. NOAA and NWS implement the act and OMB  guidance by requiring project officials to describe how planned system  and technology upgrades are linked to the agency’s programmatic  priorities and performance measures. Further, in its annual performance  plans, NOAA reports on expected NWS service improvements and  identifies the technologies and systems that are expected to help improve  them.
NWS service improvements are often expected through a combination of  system and technology improvements. For example, NWS expects to  reduce its average error in forecasting a hurricane’s path by approximately  20 nautical miles between 2005 and 2011 through a combination of  upgrades to observation systems, better hurricane forecast models,  enhancements to the computer infrastructure, and research that will be  transferred to NWS forecast operations. Also, NWS expects tornado  warning lead times to increase from 13 to 15 minutes by the end of fiscal  year 2008 after NWS completes retrofits to the NEXRAD systems, realizes  the benefits of AWIPS software enhancements, and implements new  training techniques. Table 5 provides a summary of how system upgrades  are expected to result in service improvements.
section_title: NWS’s Training Is Expected to Result in Forecast Service Improvements, but the Training Selection Process Lacks Sufficient Oversight
paragraphs: NWS provides employee training courses that are expected to help  improve forecast service performance, but the agency’s process for  selecting this training lacks sufficient oversight. Each year, NWS identifies  its training needs and develops this training in order to enhance its  services. NWS develops an annual training and education plan identifying  planned training, how this training supports key criteria, and associated  costs for the upcoming year. To develop the annual plan, program area  teams, with representatives from NWS headquarters and field offices,  prioritize and submit training recommendations. Each submission  identifies how the training will support up to eight different criteria— including the course’s effect on NWS forecasting performance measures,  NOAA strategic goals, ensuring operational continuity, and providing  customer outreach. These submissions are screened by a training and  education team, and depending on available resources, selected for  development (if not pre-existing) and implementation. The planned  training courses are then delivered through a variety of means, including  courses at the NWS training center, online training, and training at local  forecast offices.
In its 2006 training process, 25 program area teams identified 134 training  needs, such as training on how to more effectively use AWIPS, training on  an advanced weather simulator, and training on maintaining ASOS  systems. Given an expected funding level of $6.1 million, the training and  education team then selected 68 of these training needs for  implementation. NWS later identified another 5 training needs and  allocated an additional $1.25 million to its training budget. In total, NWS  funded 73 of 139 training courses.
The majority of planned training courses demonstrate a clear link to  expected forecasting service improvements. For example, NWS developed  a weather event simulator to help forecasters improve their tornado  warning lead times. In addition, AWIPS-related training courses are  expected to help improve each of the agency’s 14 forecasting performance  measures by teaching forecasters advanced techniques in using the  integrated data processing workstations.
However, NWS’s process for selecting which training courses to  implement lacks sufficient oversight. In justifying training courses,  program officials routinely link proposed courses to NWS forecast  performance measures. Specifically, in 2006, 131 of the 134 original  training needs were linked to expectations for improved forecasting  performance—including training on cardiopulmonary resuscitation, spill  prevention, leadership, systems security, and equal employment  opportunity/diversity. The training selection process did not validate or  question that these courses would improve tornado warning lead times or  hurricane warning accuracy. Although these courses are important and  likely justifiable on other bases, the overuse of this justification  undermines the distinctions among training courses and the credibility of  the course selection process. Additionally, because the training selection  process does not clearly distinguish among courses, it is difficult to  determine whether sufficient funds are dedicated to the courses that are  expected improve performance.
NWS training officials acknowledged that some of the course justifications  seem questionable and that more needs to be done to strengthen the  training selection process to ensure oversight of the justification and  prioritization process. They noted that the training division plans to  improve the training selection process over the next few years by adding a  more systematic worker-focused assessment of training needs, better  prioritizing strategic and organizational needs, and initiating post- implementation reviews. However, until NWS establishes a training  selection process that uses reliable justification and results in  understandable decisions, NWS risks selecting courses that do not most  effectively support its training goals.
section_title: Changing Concept of Operations Could Affect Nationwide Office Configuration, but Impact on Forecast Services, Staffing, and Budget Is Not Yet Known
paragraphs: NWS plans to develop a prototype of a new concept of operations—an  effort that could affect its national office configuration, including the  location and functions of its offices nationwide. However, NWS has yet to  determine many details about the impact of any proposed changes on  NWS forecast services, staffing, and budget. Further, NWS has not yet  identified key activities, timelines, or measures for evaluating the concept  of operations prototype. As a result, it is not evident that NWS will collect  the information it needs on the impact and benefits of any office  restructuring in order to make sound and cost-effective decisions.
subsections: section_title: NWS Is Evaluating Changes to Its Current Operations
paragraphs: According to agency officials, over the last several years, NWS’s corporate  board noted that the constrained budget, high labor costs, difficulty in  training and developing its employees, and a lack of flexibility in how the  agency was operating were making it more difficult for the agency to  continue to perform its mission. In August 2005, the board chartered a  working group to evaluate the roles, responsibilities, and functions of  weather offices nationwide and to make a proposal for a new concept of  operations. The group was given a set of guiding principles, including that  the proposed concept should (1) be cost effective, (2) ensure that there  would be no degradation of service, (3) ensure that weather services  nationwide were equitable, and (4) not reduce the number of forecast  offices nationwide. In addition, the working group was instructed not to  address grade structure, staffing levels, office sizes, or overall  organizational chart structure.
The group gathered input from various agency stakeholders and other  partners within NOAA and considered multiple alternatives. They  dismissed all but one of the alternative concepts because they were not  consistent with the guiding principles. In its December 2005 proposal, the  working group proposed a “clustered peer” office plan designed to  redistribute some functions among various offices, particularly when there  is a high-intensity weather event. An agency official explained that each  weather forecast office currently has a fixed geographic area for which it  provides forecasts. If a severe weather event occurs, forecast offices ask  their staff to work overtime so that there are enough personnel available  to do both the normal forecasting work and the watches and warnings  required by the severe event. If a local office becomes unable to provide  forecast and warning functions, an adjacent office will temporarily assume  those duties by calling in extra personnel to handle the workload of both  offices.
Alternatively, under a clustered peer office structure, several offices with  the same type of weather and warning responsibilities, climate, and  customers would be grouped in a cluster. Offices within a cluster would  share the workload associated with routine services, such as 7-day  forecasts. During a high-impact weather event—such as a severe storm,  flood, or wildfire—the offices would redistribute the workload to allow  the impacted office to focus solely on the event, while the other offices in  the cluster would pick up the impacted office’s routine services. In this  way, peer offices could help supplement staffing needs and the workload  across multiple offices could be more efficiently balanced.
After receiving this proposal, the NWS corporate board chartered another  team to develop a prototype of the clustered peer idea to evaluate the  benefits of this approach. The team plans to recommend the scope of the  prototype and select several weather offices for the prototype  demonstration by the end of September 2006. It also plans to conduct the  prototype demonstration in fiscal years 2007 and 2008. Initial prototype  results are due in fiscal year 2009.
section_title: Impacts of New Concept of Operations Have Yet to Be Determined
paragraphs: Many details about the impact of the changes on NWS forecast services,  staffing, and budget have yet to be determined. Sound decision making on  moving forward with a new concept of operations will require data on the  relative costs, benefits, and impacts of such a change, but at this time the  implications of NWS’s revised concept of operations on staffing, budget,  and forecasting services are unknown.
The charter for the team developing the prototype for the new concept of  operations calls for it to identify metrics for evaluating the prototype and  to define mechanisms for obtaining customer feedback. However, the  team has not yet established a plan or timeline for developing these  metrics or mechanisms. Further, it is not yet evident that these metrics  will include the relative costs, benefits, or impacts of this change or which  customers will be offered the opportunity to provide feedback. This is not  consistent with the last time NWS undertook a major change to its concept  of operations—during its modernization in the mid-1990s. During that  effort, the agency developed a detailed process for identifying impacts and  ensuring that there would be no degradation of service (see app. III for a  summary of this prior process).
Until it establishes plans, timelines, and metrics for evaluating its  prototype of a revised concept of operations, NWS is not able to ensure  that it is on track to gather the information it needs to fully evaluate the  merits of the revised concept of operations and to make sound and  informed decisions on a new office configuration.
section_title: Conclusions
paragraphs: NWS is appropriately positioning itself to improve its forecasting services  by upgrading its systems and technologies and by developing training to  enhance the performance of its professional staff. Over the next few years,  NWS expects to improve all of its 14 performance measures—ranging from  seasonal temperature forecasts, to severe weather warnings, to specialized  aviation and marine weather warnings. However, it is not clear that NWS is  consistently choosing the best training courses to improve its performance  because the training selection process does not rigorously review the  training justifications.
Recognizing that high labor costs, difficulty in training and developing its  employees, and a constrained budget environment make it difficult to  fulfill its mission, NWS is evaluating changes to its office structure and  operations in order to achieve greater productivity and efficiency. It plans  to develop a prototype of a new concept of operations that entails sharing  responsibilities among a cluster of offices. Because it is early in the  prototype process, the implications of these plans on staffing, budget, and  forecasting services are unknown at this time. However, NWS does not yet  have detailed plans, timelines, or measures for assessing the prototype. As  a result, NWS risks not gathering the information it needs to make an  informed decision in moving forward with a new office operational  structure.
section_title: Recommendations for Executive Action
paragraphs: To improve NWS’s ability to achieve planned service improvements, we  recommend that the Secretary of Commerce direct the Assistant  Administrator for Weather Services to take the following three actions:  require training officials to validate the accuracy of training justifications;    establish key activities, timelines, and measures for evaluating the  “clustered peer” office structure prototype before beginning the prototype;  and   ensure that plans for evaluating the prototype address the impact of any  changes on budget, staffing, and services.
section_title: Agency Comments
paragraphs: We received written comments on a draft of this report from the  Department of Commerce (see app. IV). In the department’s response, the  Deputy Secretary of Commerce agreed with our recommendations and  identified plans for implementing them. Specifically, the department noted  that it plans to revise its training process to ensure limited training  resources continue to target improvements in NWS performance. The  department also noted that the concept of operations working team is  developing a plan for the prototype and stated that this plan will include  the items we recommended.
The department also provided technical corrections, which we have  incorporated as appropriate.
We are sending copies of this report to the Secretary of Commerce, the  Director of the Office of Management and Budget, and other interested  congressional committees. Copies will be made available to others on  request. In addition, this report will be available at no charge on our Web  site at www.gao.gov.
If you have any questions about this report, please contact me at (202) 512- 9286 or by e-mail at pownerd@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were (1) to evaluate the National Weather Service’s (NWS)  efforts to achieve improvements in the delivery of its services through  upgrades to its systems, models, and computational abilities; (2) to assess  the agency’s plans to achieve improvements in the delivery of its services  through the training and professional development of its employees; and  (3) to evaluate the agency’s plans for revising its nationwide office  configuration and the implications of these plans on local forecasting  services, staffing, and budgets.
To evaluate NWS’s efforts to achieve service improvements through  system and technology upgrades, we reviewed the agency’s system  development plans and discussed system-specific plans with NWS  program officials. We assessed system-specific documentation justifying  system upgrades to evaluate whether these upgrades were linked to  anticipated improvements in performance goals. We also evaluated NWS  performance goals and identified the extent to which anticipated service  improvements were tied to system and technology upgrades. We  interviewed National Oceanic and Atmospheric Administration (NOAA)  and NWS officials to obtain clarification on agency plans and goals.
To assess NWS’s plans for achieving service improvements through the  training and professional development of its employees, we reviewed NWS  policies and plans for training and professional development. We reviewed  the agency’s service performance goals and assessed the link between  those goals and planned and expected training and professional  development activities. We also interviewed NWS officials responsible for  training and professional development activities.
To evaluate the status and potential impact of any plans to revise the  national office configuration, we assessed studies of options for changing  the NWS concept of operations. We also reviewed the charter for the  prototype and interviewed key NWS officials to determine the possible  effect of these plans on local forecasting services, staffing, and budgets  and to identify plans for determining the implications of changing to a new  concept of operations.
We performed our work at NWS headquarters in the Washington, D.C.,  metropolitan area, and at geographically diverse NOAA and NWS weather  forecast offices in Denver and in Tampa, and at the NWS National  Hurricane Center in Miami. We performed our work from October 2005 to  June 2006 in accordance with generally accepted government auditing  standards.
section_title: Appendix II: NWS Performance Goals for Fiscal Years 2005 to 2011
paragraphs: A measure of the difference between  the projected locations of the center  of storms and the actual locations in  nautical miles for the Atlantic Basin    miles, respectively, and ceilings and visibilities are greater than, or eual to, 500 feet and/or 1 mile,  respectively.
section_title: Appendix III: NWS Previously Used A Stringent Process to Ensure Service Was Not Degraded
paragraphs: In the 1980s, NWS began a nationwide modernization program to upgrade  weather observing systems such as satellites and radars, to design and  develop advanced computer workstations for forecasters, and to  reorganize its field office structure. The goals of the modernization were to  achieve more uniform weather services across the nation, improve  forecasting, provide more reliable detection and prediction of severe  weather and flooding, achieve higher productivity, and permit more cost- effective operations through staff and office reductions.
NWS’s plans for revising its office structure were governed by the Weather  Service Modernization Act, which required that, prior to closing a field  office, the Secretary of Commerce certify that there was no degradation of  service. NWS developed a plan for complying with the law. To identify  community concerns regarding modernization changes and to study the  potential for degradation of service, the Department of Commerce  published a notice in the Federal Register requesting comments on service  areas where it was believed that services could be degraded by planned  modernization changes. The department also contracted for an  independent assessment by the National Research Council on whether  weather services would be degraded by the proposed changes. As part of  this assessment, the contractor developed criteria to identify whether  service would be degraded in certain areas of concern. The department  then applied these criteria to areas of concern to determine whether  services would be degraded or not. Before closing any office, the Secretary  of Commerce certified that services would not be degraded.
section_title: Appendix IV: Comments from the Department of Commerce
section_title: Appendix V: GAO Contact and Staff Acknowledgments
paragraphs: David A. Powner, (202) 512-9286 or pownerd@gao.gov.
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, William Carrigg, Barbara Collier,  Neil Doherty, Kathleen S. Lovett, Colleen Phillips, Karen Talley, and  Jessica Waselkow made key contributions to this report.