Department of Commerce: National Weather Service Modernization and NOAA Fleet Issues
Published: 1999-02-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO discussed the: (1) status of the National Weather Service (NWS) systems modernization; and (2) most cost-effective alternatives for acquiring the National Oceanic and Atmospheric Administration's (NOAA) marine data.
section_title: What GAO Found
paragraphs: GAO noted that: (1) although NWS is nearing completion of its systems modernization effort, two significant challenges face it this year: (a) deploying the final system of modernization; and (b) ensuring that all of its mission-critical systems are year 2000 compliant; (2) NWS has made progress on both fronts; (3) in the NOAA fleet area, NOAA now outsources for more of its research and data needs but plans to spend $185 million over the next five years to acquire four new replacement NOAA fisheries research ships; and (4) thus, GAO believes that continued congressional oversight of this area, as well as NOAA's budget requests for replacement or upgraded ships, is needed to ensure that NOAA is pursuing the most cost-effective alternatives for acquiring marine data.

section_title: Letter
paragraphs: We appreciate the opportunity to join in today’s hearing to discuss our  work in two areas under the responsibility of the National Oceanic and  Atmospheric Administration (NOAA), a component of the Department of  Commerce.  At your request, we will first discuss the status of the National  Weather Service (NWS) systems modernization and then address the most  cost-effective alternatives for acquiring NOAA’s marine data.  Last month,  we continued to designate the NWS systems modernization effort as a high- risk information technology area and the NOAA fleet as an additional major  management challenge confronting the Department of Commerce.
In brief, although NWS is nearing completion of its systems modernization  effort, two significant challenges face it this year—deploying the final  system of the modernization and ensuring that all of its mission-critical  systems are Year 2000 compliant.  NWS has made progress on both fronts.   In the NOAA fleet area, NOAA now outsources for more of its research and  data needs but plans to spend $185 million over the next 5 years to acquire  four new replacement NOAA fisheries research ships.  Thus, we believe  that continued congressional oversight of NOAA’s budget requests for  replacement or upgraded ships is needed to ensure that NOAA is pursuing  the most cost-effective alternatives for acquiring marine data.
subsections: section_title: National Weather Service Modernization
paragraphs: In the 1980s NWS began a nationwide modernization program to upgrade  observing systems such as satellites and radars, and design and develop  advanced computer workstations for forecasters.  The goals of the  modernization are to achieve more uniform weather services across the  nation, improve forecasting, provide better detection and prediction of  severe weather and flooding, permit more cost-effective operations  through staff and office reductions, and achieve higher productivity.  For  example, NWS plans to reorganize its field office structure from 256 offices  (52 Weather Service Forecast Offices and 204 Weather Service Offices), to  121.  As of February 1999, NWS officials told us that 132 offices have been  closed.
NWS’ system modernization includes four major systems development  programs, which are expected to collectively cost about $4.5 billion.  I  would like to briefly describe each.
Next Generation Weather Radar (NEXRAD).  This is a program to acquire  166 Doppler radars.  Largely deployed, these radars have helped NWS  increase the accuracy and timeliness of warnings for severe  thunderstorms, tornadoes, and other hazardous weather events.  The  reported cost of this program is just under $1.5 billion.
Next Generation Geostationary Operational Environmental Satellite  (GOES-Next).  This is a program to acquire, launch, and control five  geostationary satellites, GOES-I through GOES-M, which assist in the  mission of identifying and tracking severe weather events, such as  hurricanes.  The first satellite in the current series was launched in 1994  and the fifth is scheduled for launch in 2002.  The total cost for these five  satellites, including launch services and ground systems, is estimated to be  just under $2 billion.
Automated Surface Observing System (ASOS).  This is a program to  automate and enhance methods for collecting, processing, displaying, and  transmitting surface weather conditions, such as temperature and  precipitation.  The system is planned for installation at 314 NWS locations.   Estimated costs for the ASOS Program are about $350 million, which  includes the NWS units and another 679 units for the Federal Aviation  Administration and the Department of Defense.
Advanced Weather Interactive Processing System (AWIPS).  This program  integrates, for the first time, satellite, radar, and other data to support  weather forecaster decision-making and communications; it is the linchpin  of the NWS modernization.  AWIPS, which was originally scheduled to be  developed incrementally in a series of six modules, or builds, is currently  set to be deployed to 152 locations after the fourth build by the end of June  1999.
In 1995 we designated the NWS modernization a high-risk area for the  federal government because of its estimated $4.5 billion cost, its  complexity, its criticality to NWS’ mission of helping to protect life and  property through early forecasting and warnings of potentially dangerous  weather, and its past problems--documented in several of our reports.  Our  1997 high-risk series reported that although the development and  deployment of the observing systems associated with the modernization  were nearing completion, unresolved issues remained.  These concerned  the systems' operational effectiveness and efficient maintenance.  For  example, new radars were not always up and running when severe weather  threatened, and ground-based sensors fell short of user expectations,  particularly during active weather.  We recommended that NWS correct  shortfalls in radar performance, and define and prioritize all ground-based  sensor corrections according to user needs.
Some of our radar and ground-based sensor performance concerns were  addressed, while others remain.  We recently reported that a NEXRAD unit  in southern California failed to consistently meet NWS’ own NEXRAD  availability requirement, and recommended that the Weather Service  correct the problem such that the radar meets availability requirements.  NWS agreed, and has several activities planned to bring about such  improvement.
While there have been specific performance problems, NWS reports that  the new radars and satellites overall have enabled it to generate better data  and greatly improved forecasts and warnings.  We continue to view the  NWS modernization as a high-risk area, however, for two primary reasons:   (1) NWS lacks an overall architecture to guide systems development and  (2) the final piece of the modernization--AWIPS (the forecaster  workstations that will integrate weather data from NEXRAD, GOES-Next,  and ASOS)--has not yet been deployed.  At this point I would like to  discuss these issues in more detail.
subsections: section_title: NWS Modernization Lacks Overall Systems Architecture
paragraphs: A systems architecture is an essential tool for guiding effective and  efficient systems development and evolution.  We initially reported in 1994 that the NWS modernization needed such an overall technical blueprint;  NWS agrees--and is currently working on one.  Until such an architecture is  developed and enforced, the modernization will continue to be subject to  higher costs and reduced performance.  This is an important point as  component systems continue to evolve to meet additional demands and  take advantage of improved technology.  The Assistant Administrator for  Weather Services shares this view, and said recently that NWS plans to  intensify its efforts to develop a systems architecture.
section_title: Delivery of AWIPS Remains a Concern
paragraphs: Until AWIPS is fully deployed and functioning properly, NWS will not be  able to take full advantage of the $4.5 billion total investment it has made in  the modernization.  Over the past several years, we have reported that  AWIPS has encountered delays and cost increases due to design problems  and management shortcomings and have made several recommendations  to improve management of this critical component of the modernization.  NWS has acted on most of our recommendations.  I would like to now  update you on AWIPS’ cost, schedule, software development, and  maintenance.
The cost to develop AWIPS was estimated at $350 million in 1985; a decade  later, that figure had risen to $525 million.  However, in testimony and a  report issued in 1996, we pointed out the inaccuracy of this $525 million  estimate due to the omission of several cost factors, including known  contract increases.  The Department of Commerce later committed to a  $550 million funding cap.  Yet as we testified in April 1997, it would prove  extremely difficult for NOAA to develop and deploy AWIPS within the  $550 million cap if any problems were encountered.   Given the size and  complexity of the development--and recognizing that even managed risks  can turn into real problems--we testified that such problems were likely to  occur and that costs would likely exceed $550 million.
In accordance with a recommendation we made in 1996, the department  contracted for an independent cost estimate of AWIPS because of the  uncertainty about whether it could be delivered within the $550 million cap  given the increased software development expenses. According to the  assessment dated February 2, 1998, the likely cost to complete AWIPS  through its final build--build 6--was $618 million.
In March 1998, we reported that although AWIPS was planned for full  deployment through build 6 in 1999--at 152 locations nationwide--that  schedule is now in doubt.  The latest schedule calls only for build 4-- actually build 4.2--to be completed in June, within the $550 million cap.   Also as we testified last year, completion dates for builds 5 and 6 were  uncertain because NWS wanted to ensure that requirements for those  modules were not extraneous to mission needs, in order to minimize future  cost increases.  This reflects a recommendation we made in 1996 for all  AWIPS builds.  In August 1998, an independent review team reported that  build 5 requirements are essential to NWS’ core mission and that the cost to  complete should range from an additional $20 to $25 million above the  $550 million cap.  The team concluded that build 6 requirements should not  be pursued, however, because they “resemble capabilities desired, rather  than requirements.”
According to the AWIPS program manager, deployment of build 4.2 will  result in improved forecasts and warnings, a reduction of 106 staff, and the  decommissioning of the current Automation of Field Operations and  Services (AFOS) system.  The program manager added that build 5 will be  pursued in order to realize expected further improvements in weather  forecasts and warnings, a reduction of an additional 69 staff, and the  decommissioning of the NEXRAD workstations.  Schedules for build 5  have not yet been developed.  To help ensure that build 4.2 will be delivered  within the cap, the Assistant Administrator for Weather Services has  contracted with an independent accounting firm to verify program  expenditures.
The most critical risk factors underlying questions about AWIPS' future  relate to software development.  We have frequently reported on this and  made several recommendations to improve AWIPS’ software development  processes.  Software quality is governed largely by the quality of the  processes used to develop it; however, NWS’ efforts to develop AWIPS  software have lacked defined development processes.  Such processes are  all the more essential because of NWS’ increased use of software code  developed internally at NOAA’s Forecast Systems Laboratory (FSL) in  Boulder, Colorado--a research and development facility that primarily  develops prototype systems.  This software code has not been developed  according to the rigorous processes commonly used to develop production- quality code.  Failure to adhere to these processes may result in unstable  software that will continue to cause cost increases and schedule delays.
The cost assessment delivered in February 1998 also found risk inherent in  the development of builds 4 through 6 because of the transitioning of FSL- developed software to AWIPS and the uncertainty surrounding  requirements for these builds.  NWS officials have acknowledged these  software development process weaknesses, and have told us that they  continue to strengthen these processes.  For example, NWS reports that all  AWIPS software, both that developed by the government and the  contractor, is being controlled under a common configuration management  process.
Another risk area concerns the network control facility, which provides the  ability to monitor and maintain AWIPS sites across the country from a  single location.  As we testified last year, through build 3, AWIPS was still  experiencing difficulty with the central location's ability to detect and  respond to problems.  We further testified that since these problems  concerned only a limited number of sites that as more sites come on line,  problems can be expected to increase.  NWS officials have acknowledged  that the poor performance of the network control facility continues to be a  prime concern, have sought the advice of external consultants, and have  initiated a number of actions to improve performance of this facility.
section_title: Progress Made on Year 2000, But Critical Testing and Contingency Planning Efforts Remain
paragraphs: Finally, a critical risk area is whether the AWIPS builds--and, indeed, all  modernization components--will be Year 2000 compliant.  AWIPS to date  is not Year 2000 compliant.  Build 4.2--set for completion this June--is  intended to make all AWIPS applications Year 2000 compliant.  In the event  it is late, NWS has renovated its current system, Automation of Field  Operations and Services, to be ready as a potential backup.
Yet even if Year 2000 compliance ceases to be an issue with build 4.2, NWS’  companion modernization systems will need to be compliant because of  the amount of data they exchange.  NWS reports that five of the six  mission-critical systems that interface with AWIPS are already Year 2000  compliant, on the basis of individual systems tests.  The remaining system  is scheduled to be compliant by March 31, 1999, according to the  Department of Commerce’s February 1999 Quarterly Year 2000 Progress  Report to the Office of Management and Budget.
To ensure that these mission-critical systems can reliably exchange data  with other systems and that they are protected from errors that can be  introduced by external systems, NWS has begun to perform end-to-end  testing.  These tests include multiple Weather Service systems working  together and critical interfaces with the Department of Defense and the  Federal Aviation Administration.  NWS plans to continue to conduct this  end-to-end testing through March of this year.  The final report on the  results of these tests is scheduled to be issued this May.
We suggest that NWS consider conducting additional end-to-end testing  after the final version of AWIPS is delivered, which is currently scheduled  for this June. Currently, NWS is using a prior version of AWIPS in its end-to- end testing—a version that continues to be modified as AWIPS’ system- level testing progresses.  Testing with the final version of AWIPS will help  to ensure that the production system that will be running in the year 2000  will work with its interrelated systems.
To reduce the risk and potential impact of Year 2000-induced information  systems failures on the Weather Service’s core business processes, it is  critical that NWS have contingency plans in place that will help ensure  continuity of operations through the turn of the century. Without such  plans, NWS will not have well-defined processes to follow in the event of  failures.  NWS depends on data provided by other federal agencies as well  as on services provided by the public infrastructure (e.g., power, water,  voice and data telecommunications).  One weak link anywhere in this chain  of critical dependencies could cause major disruption to NWS operations.   Given these interdependencies, it is imperative that contingency plans be  developed for all critical core business processes.  According to NWS’ Year  2000 program manager, the Weather Service has begun drafting  contingency plans for three core business processes:  those that  (1) observe weather data, (2) produce forecasts and warnings, and  (3) disseminate data.  It is essential that NWS develop these business  continuity and contingency plans expeditiously, and test these plans to  ensure that they are capable of providing the level of support needed to  allow continued functioning of NWS’ core business processes in the event  of failure.
As noted in our business continuity and contingency guide, another key  element of such a plan is the development of a zero day or day one risk  reduction strategy and, more generally, procedures for the period between  December 1999 and early January 2000.  Key aspects of this strategy can  include the implementation of (1) an integrated control center, whose  purposes include the internal dissemination of critical data and problem  management and (2) a timeline that details the hours in which certain  events will occur (such as when backup generators will be started) during  the late December and early January rollover period.  To date, NWS has no  such strategy.  We suggest that the development of such a risk reduction  strategy be undertaken.
In conclusion, NWS has made progress on the development and  operational testing of the forecaster workstations and its Year 2000 testing  and contingency planning.  However, cost, schedule, and technical risks  associated with the workstations continue to be concerns.  Further, the  results of NWS’ Year 2000 end-to-end testing and business continuity and  contingency plans are expected to be delivered soon.
section_title: Pursuing the Most Cost-Effective Alternatives for Acquiring NOAA’s Marine Data
paragraphs: NOAA has an aging in-house fleet of 15 ships that are used to support its  programs in fisheries research, oceanographic research, and hydrographic  charting and mapping.  Most of NOAA’s ships are past their 30-year life  expectancies and many of them are costly and inefficient to operate and  maintain and lack the latest state-of-the-art technology.  NOAA’s ships are  managed and operated by a NOAA Corps of about 240 uniformed service  commissioned officers who, like the Public Health Service Corps, perform  civilian rather than military functions but are covered by a military-like pay  and benefits system.
For more than a decade, congressional committees, public and private  sector advisory groups, the National Performance Review, the Commerce  Office of Inspector General (OIG), and our office have urged NOAA to  aggressively pursue cost-effective alternatives to its in-house fleet of ships.  We have also reported and testified on issues relating to NOAA’s  Commissioned Corps that manages and operates the in-house fleet of  ships.
We reported on NOAA’s fleet operations and fleet modernization needs in  1986 and again in 1994 and summarized our earlier work, the Commerce  OIG’s work, and the Department of Commerce’s corrective actions in  summary reports in January 1998 and January 1999.  As part of our recent  special performance and accountability series of reports, we identified the  NOAA fleet as one of four major performance and management issues  confronting the Department of Commerce.  As early as 1986 we reported  that NOAA needed to develop more definitive information on private ships’  availability, capability, and costs before taking any action to deactivate  NOAA’s ships.  In 1994, we reported that NOAA (1) lacked the financial  and operational data it needed to adequately assess whether chartered and  contracted ships could cost effectively meet the needs of its programs and  (2) had no assurance that its fleet modernization plan represented the most  cost effective means of meeting future program requirements.   Consequently, we recommended that NOAA take several actions to ensure  that all viable and cost-effective options for accomplishing its program  missions are considered in making decisions on future fleet modernization.
The Commerce OIG has also reported and testified several times on the  NOAA fleet modernization issue, identified the fleet as one of the top 10  management problems facing the Department of Commerce in April 1997,  January 1998, and again in December 1998, and continues to believe that  NOAA could and should be doing more to pursue cost-effective alternatives  to its in-house fleet of ships for acquiring marine data.  Following reports  by us, the Commerce OIG, and others, the Department of Commerce  initially identified the NOAA fleet as a material weakness in its annual  Federal Managers’ Financial Integrity Act (FMFIA) report for fiscal year  1990.  It remains a material weakness today.
Since 1990, NOAA has developed several fleet replacement and  modernization plans that call for investments of hundreds of millions of  dollars to upgrade or replace these ships, and each has been criticized by  the Commerce OIG for not pursuing alternative approaches strongly  enough.  For example, in a 1996 program evaluation report on NOAA’s  $1 billion 1995 fleet modernization plan, the OIG recommended that NOAA  terminate its fleet modernization efforts; cease investing in its ships;  immediately begin to decommission, sell, or transfer them; and contract for  the required data or ship services.
In response to these criticisms, NOAA now says that it has taken steps to  improve the cost efficiency of its fleet and significantly increased its  outsourcing for these services from about 15 percent in 1990 to over 40 percent today.  According to NOAA, for example, it has removed seven  ships from service and brought one new and two converted Navy ships into  service since 1990, now outsources for about 46 percent of its research and  survey needs, and expects to further increase its use of outsourcing to  about 50 percent over the next 10 years.
Although NOAA apparently has made progress in reducing the costs of its  fleet and outsourcing for more of its research and data needs, NOAA  continues to rely heavily on its in-house fleet and still plans to replace or  upgrade some of these ships.  In this regard, the President’s budget for  fiscal year 2000 proposes $52 million for construction of a new fisheries  research ship and indicates that NOAA plans to spend a total of $185  million for four new replacement ships over the 5-year period ending in  fiscal year 2004--$52 million in 2000, $51 million in 2001, $40 million in 2002,  $40 million in 2003, and $2 million in 2004.  We have not had an opportunity  to review the latest studies of NOAA’s fleet modernization efforts or NOAA’s  acquisition plan for its fisheries research mission.  Thus, we do not know  whether or not NOAA’s proposed replacement ships are the most cost- effective alternative currently available for meeting these fisheries research  needs.
In addition to its proposed acquisitions, NOAA also continues to repair and  upgrade its aging fleet of existing ships.  Since 1990, it has repaired and  upgraded seven of its existing ships and plans to repair and upgrade two  more in 1999.  According to the President’s recent budget requests, NOAA  spent $12 million in 1996 and $13 million in 1997 to modernize, convert, and  replace its existing ships.  Also, it spent $21 million on fleet maintenance  and planning in 1998 and expects to spend $13 million in 1999 and  $9 million in 2000.
The question of the viability of the NOAA fleet is entwined with the issue of  the NOAA Corps, which operates the fleet.  In 1995, the National  Performance Review, noting that the NOAA Corps was the smallest  uniformed service and that the fleet it commanded was obsolete,  recommended that the NOAA Corps be gradually reduced in numbers and  eventually eliminated.  We reported in October 1996 that the NOAA Corps  generally does not meet the criteria and principles cited by the Department  of Defense for a military compensation system.  We also noted that other  agencies, such as the Navy, the Environmental Protection Agency (EPA),  and the Federal Emergency Management Agency (FEMA), use federal  civilian employees or contractors to carry out duties similar to the  functions that NOAA assigns to the Corps.  Commerce developed a plan  and legislative proposal to “disestablish” or civilianize the NOAA Corps in  1997, but the Congress did not adopt this proposal.
According to NOAA and to the Department of Commerce’s annual  performance plans for fiscal years 1999 and 2000 under the Results Act, the  NOAA Corps has been downsized from over 400 officers in fiscal year 1994  to about 240 at the beginning of fiscal year 1999, achieving gross annual  cost savings of at least $6 million.  In June 1998, NOAA announced a new  restructuring plan for the NOAA Corps.  NOAA’s plan focused on the need  for a NOAA Commissioned Corps of about 240 officers.  NOAA’s June 1998  restructuring plan also called for a new civilian director of the NOAA Corps  and a new recruiting program.
However, the Congress had other ideas.  The Omnibus Appropriations Act  for fiscal year 1999 set the number of NOAA Corps officers at 250.   Subsequently, the Governing International Fishery Agreement Act (Public  Law 105-384, approved November 13, 1998) made other changes in NOAA’s  proposed restructuring plan.  This act authorized a NOAA Corps of at least  264 but not more than 299 commissioned officers for fiscal years 1999  through 2003, requires that a uniformed flag officer be the NOAA Corps’  operational chief, and directed the Secretary of Commerce to lift the then- existing recruiting freeze on NOAA Corps officers.  According to the NOAA  Corps, it expects to have about 250 commissioned officers by the end of  fiscal year 1999.
In summary, NWS faces significant challenges this year—both in deploying  the initial version of AWIPS and in addressing the Year 2000 problem.   Longer term, NWS still needs to develop an overall systems architecture  and to develop AWIPS’ build 5 requirements since they are essential to  NWS’ core mission.  In the NOAA fleet area, continuing congressional  oversight of NOAA’s budget requests for replacement or upgraded ships is  needed to ensure that NOAA is pursuing the most cost-effective  alternatives for acquiring marine data.
This concludes our statement.  We would be happy to respond to any  questions that you or other members of the Subcommittee may have at this  time.
The first copy of each GAO report and testimony is free. Additional copies are $2 each.  Orders should be sent to the  following address, accompanied by a check or money order made  out to the Superintendent of Documents, when necessary, VISA and  MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address are  discounted 25 percent.
U.S. General Accounting Office P.O. Box 37050 Washington, DC  20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and  testimony.  To receive facsimile copies of the daily list or any list  from the past 30 days, please call (202) 512-6000 using a touchtone  phone.  A recorded menu will provide information on how to obtain  these lists.