Since the early 1980s, NWS has been modernizing its observing, information processing, and communications systems to improve the accuracy, timeliness, and efficiency of weather forecasts and warnings. The $4.5 billion modernization includes four major systems—AWIPS, the Next Generation Geostationary Operational Environmental Satellites (GOES-Next), the Next Generation Weather Radars (NEXRAD), and the Automated Surface Observing System (ASOS). GOES-Next, NEXRAD, and ASOS form the foundation of NWS’ observing infrastructure, watching the weather from earth and space. The two orbiting GOES-Next satellites each provide as many as 1,200 digital weather images daily to each forecast office, compared to less than 100 from their predecessors. The almost 150 NEXRADs are to blanket the continental United States and using Doppler radar technology, allow forecasters to see inside weather events to detect motion and dynamics that were invisible to pre-NEXRAD radars. ASOS, which consists of nearly 900 separate ground-based sensor sets, is to provide a portfolio of weather readings, such as temperature and visibility, from more locations and with greater frequency than has been done by human observers. AWIPS is to function as the “central nervous system” of a modernized NWS. That is, it is to be the information processing and display system that forecasters use to integrate, analyze, and graphically view the immense number of weather observations and products that form the basis for each day’s weather and river forecasts and warnings. It is also to be the national communications infrastructure for NWS’ many forecasting offices and centers, connecting them not only to each other but also linking them to the many users of forecasts and warnings throughout the nation. Using AWIPS’ advanced processing, display, and communications capabilities, NWS expects to fully capitalize on its new observing systems for the first time. Without AWIPS, these observing systems cannot be maximized because the current Automation of Field Operations and Services (AFOS) computer and communications system and the associated weather office display systems cannot accommodate the mountainous data streams that these observing systems now provide. For example, the current system can only accept two satellite images an hour. With AWIPS, GOES-Next images are to be received in real time, meaning that up to eight images can be received and displayed each hour. According to NWS, AWIPS is to contribute to improvements in the accuracy and timeliness of forecasts and warnings as well as streamlining its operations and downsizing its organization. All told, NWS expects that the combined pieces of the modernization will result in the number of NWS field offices dropping from 254 to 119 and the number of NWS staff falling from 5,100 to 4,678. AWIPS is expected to improve forecaster productivity by allowing forecasters to view disparate data sets in an integrated fashion, perform an assortment of scientific computations on these data sets, and graphically display and interact with these data sets. Currently, such activities are generally performed manually as data from the various observing systems are displayed on multiple screens and assimilated by the forecaster. For example, today, when forecasters want to combine radar and satellite images to view weather pattern movements, the images must be manually overlayed on transparencies. With AWIPS, such integration is expected to occur automatically on the AWIPS workstation with a few simple mouse clicks or keystroke commands. In short, AWIPS is to result in forecasters spending less time physically and mentally manipulating data and more time practicing meteorology. The National Oceanic and Atmospheric Administration (NOAA), which is NWS’ parent agency, began AWIPS in the mid-1980s to replace AFOS. Since then, NWS has invested considerable time and effort in analyzing and defining AWIPS’ requirements, effectively involving users, as we reported in 1993, through hands-on experience with prototypes. In 1992, NOAA awarded the AWIPS development contract to the Planning Research Corporation (PRC). Because of the contractor’s failure to deliver an acceptable AWIPS design, NWS renegotiated the contract in 1995, basically assuming responsibility for development of all hydrology and meteorology application software and assigning the contractor responsibility for delivering the hardware and systems software and for integrating the entire system. To fulfill its responsibility, NWS established joint NWS and contractor applications software (for example, the software that executes atmospheric and hydrological numerical and statistical models, manipulates satellite and radar graphics, etc.) development teams. NWS’ current plans call for building and integrating AWIPS in seven increments. Thus far, NWS and the contractor have installed a very limited version of AWIPS, the first increment, at three sites to gain some experience in developing, testing, implementing, and operating a limited capability AWIPS. This limited version is also intended to validate selected AWIPS architectural features, such as satellite broadcasts and the central communications and system monitoring “hub.” NWS has begun development of the second increment. NWS estimates that AWIPS will cost $525 million to fully develop and deploy. Deployment of less than the full AWIPS capability to NWS field offices and national centers is now scheduled to begin in 1996. Full AWIPS deployment is scheduled to begin in 1999. AWIPS consists of about 22,000 requirements that have been grouped into about 450 higher-level capabilities. These capabilities are described in the AWIPS System/Segment Specification, commonly referred to as the “A-Spec,” which relates about three-fourths of the capabilities to five broad functional areas. The five functional areas are (1) communications, (2) monitoring and control, (3) processing, (4) display and interaction, and (5) data management. The first two functional areas constitute what NWS calls the AWIPS network segment (that is, the national communications infrastructure). The latter three are referred to by NWS as the AWIPS site segment (that is, the functionality applicable to AWIPS sites). Appendix II provides examples of AWIPS capabilities for each functional area. The remaining one-fourth relate to such capabilities as AWIPS’ performance, security, availability, and flexibility. Figure 1 depicts this AWIPS hierarchy. Investments in information technology, like AWIPS, should be justified on the basis of whether or not all planned system capabilities will make a clear difference in advancing mission efficiency and effectiveness (for example, improved forecasts and warnings). NWS has not done this for AWIPS. According to NWS officials, they have not explicitly linked either AWIPS requirements or higher-level capabilities to mission improvements, and they have no plans to do so because they claim that other requirements reviews, analyses, and validation activities already provide implicit justification for all AWIPS’ proposed capabilities. We disagree. We carefully reviewed these other activities and while we found them to be valuable for different reasons, they were neither intended to nor do they demonstrate that AWIPS’ full array of capabilities will improve NWS mission effectiveness. As a result, NWS risks unnecessarily spending money on AWIPS capabilities that do not satisfy any of its mission improvement goals—better forecasts, fewer field offices, and fewer staff. Office of Management and Budget (OMB) Circular A-130, Management of Federal Information Resources, requires agencies to create and maintain management and technical frameworks that define linkages between mission needs and information technology capabilities. OMB Circular A-109, “Major System Acquisitions,” expands on A-130, requiring federal agencies to make system design decisions based on a review of proposed system functional and performance capabilities contributions to mission needs and program objectives. In effect, agencies developing computer systems, like AWIPS, are to show that proposed system capabilities will produce some mission effectiveness or efficiency gain, like more reliable and timely forecasts or office and staffing reductions. These requirements are consistent with our recent findings on how leading public and private organizations tie technology investments to measurable mission improvements. We found that successful organizations’ information system investment decisions are tied to explicit and quantifiable mission improvements. By doing so, these organizations know that investing in system requirements will make a difference in mission outcomes. Ensuring that proposed system capabilities are justified before expensive software development begins requires validating (that is, proving) that system requirements are anchored in user needs, which in turn are grounded in positive mission impacts. To do less increases the chances of spending money on capabilities that, even though desired by users, will not advance the organization’s effectiveness or efficiency. Accordingly, software development guidance advocates assuring traceability from derived system requirements, designs, and implementations to both original user needs and mission needs. Approaches to validating proposed capabilities include performance modeling and prototyping. According to NWS, planned AWIPS capabilities are necessary and will contribute to NWS’ goals of weather forecast and warning improvements, field office consolidation, and staff reductions. However, NWS officials were unable to produce any analysis or associated documentation to validate this claim. Instead, they presented the results of past AWIPS requirements analysis and definition activities and discussed ongoing requirements validation activities that, while useful in their own right, do not justify AWIPS capabilities on the basis of mission improvements. Each of these requirements review activities are discussed below. The first AWIPS requirements review began in 1984, when NWS initiated the Denver AWIPS Risk Reduction and Requirements Evaluation (DARE) program, an extensive AWIPS prototyping effort to analyze and refine meteorology requirements. Later, NWS augmented the DARE prototype addressed roughly half of AWIPS’ 22,000 requirements. The half that were addressed equates to over three-fourths of AWIPS’ total lines of code. Moreover, even those AWIPS capabilities that were part of the prototypes, with one limited exception, were not explicitly linked to measurable improvements in NWS’ mission effectiveness during these prototyping activities. The one exception is an NWS analysis linking DARE activities occurred at the Norman, Oklahoma, weather office to emulate the operations of a future, modernized weather office.