Aviation Weather: Services at Key Aviation Facilities Lack Performance Measures, but Improvement Efforts Are Under Way
Published: 2008-02-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The National Weather Service (NWS), an agency under the Department of Commerce's National Oceanic and Atmospheric Administration (NOAA), provides staff on-site at each of the Federal Aviation Administration's (FAA) en route centers--the facilities that control high-altitude flight outside the airport tower and terminal areas. This group of NWS meteorologists provides air traffic managers with forecasts and briefings on regional conditions such as turbulence and icing. Over the last few years, FAA has been exploring options for enhancing the efficiency of the aviation weather services provided by these NWS meteorologists. In late December 2007, FAA delivered revised requirements and associated performance measures to NWS to improve these services. GAO was asked to summarize key segments of its report being released today, including its assessment of NWS and FAA efforts to ensure the quality of aviation weather services at en route centers, and its recommendations to improve these efforts. In addition, GAO was asked to provide an update on FAA's recent efforts to establish aviation weather requirements and performance measures, and NWS's plans for responding to these requirements. To do so, GAO summarized segments of its report, reviewed FAA's recently released requirements, and interviewed the official responsible for NWS's response.
section_title: What GAO Found
paragraphs: Although interagency agreements between NWS and FAA state that both agencies have responsibilities for assuring and controlling the quality of aviation weather observations, neither agency consistently does so for weather products and services produced at the en route centers. Specifically, neither agency has developed and implemented performance measures and metrics, regularly evaluated weather service unit performance, or provided feedback to improve these aviation weather products and services. Because of this lack of performance tracking and oversight, NWS cannot demonstrate the quality or value of its services, and FAA cannot ensure the quality of the services it funds. Until both agencies are able to measure and ensure the quality of the aviation weather products at the en route centers, FAA may not be getting the information it needs to effectively manage air traffic. In its report being issued today, GAO is making recommendations to the Secretaries of Commerce and Transportation to ensure that NWS and FAA develop performance measures, evaluate the services against those measures, and provide feedback to NWS. Commerce agreed with the recommendations and stated that NOAA would work with FAA to develop methods for performance monitoring and evaluation. Transportation did not agree or disagree with the recommendations, but stated that FAA's revised requirements would establish performance measures and evaluation procedures, and that FAA would negotiate with NWS to implement them. FAA has begun to address GAO's recommendations. In late December 2007, FAA finalized its new requirements, including performance measures and methods for evaluating performance and providing feedback to NWS. In doing so, FAA provides its overall vision for aviation weather services, reiterates its need for existing products and services, provides revisions to existing requirements, and defines a new product. FAA directed NWS to respond by May 2008 and include plans in its response for three operational concepts--in its existing configuration located at the 21 en route centers, through remote services provided by a reduced number of regional facilities, and through remote services provided by a single centralized facility. FAA stated that NWS should assume a transition time of 90 days for the existing configuration, 180 days for regionalized services, and 1 year for a single facility. NWS plans to respond to FAA by the May 2008 deadline, but FAA's estimated time frames for transitioning to a new operational concept may be overly ambitious. Given the importance of accurate and timely weather information in air traffic control, it will be important for NWS to conduct a thorough evaluation before it transitions to a new operational concept in order to ensure that there are no impacts on the continuity of air traffic operations and no degradation of weather service.

section_title: Letter
paragraphs: We appreciate the opportunity to participate in today’s hearing to discuss  our work on the National Weather Service’s (NWS) aviation weather  services. NWS is responsible for providing storm and flood warnings and  weather forecasts for the United States, its territories, and adjacent oceans  and waters. NWS’s weather products are also a vital component of the  Federal Aviation Administration’s (FAA) air traffic control program,  providing weather information to local, regional, and national air traffic  management, navigation, and surveillance systems. NWS aviation weather  products include forecasts and warnings of meteorological conditions that  could affect air traffic, including thunderstorms, air turbulence, and icing.  In addition to providing aviation weather products developed at its own  facilities, NWS also provides staff on-site at each of FAA’s en route  centers—the facilities responsible for controlling high-altitude air traffic  outside the tower and terminal areas.
Over the last few years, FAA has been exploring options for enhancing the  efficiency of the aviation weather services provided at its en route centers.  In September 2005, the agency asked NWS to restructure its services to be  more efficient; in response, NWS conducted a prototype and proposed  restructuring its offices to provide services remotely. FAA declined this  proposal in favor of making its existing requirements more precise. In late  December 2007, FAA delivered its revised requirements to NWS to  improve these services.
As requested, our testimony summarizes our report being released today  on NWS’s aviation weather services and provides an update on recent  efforts to develop aviation weather requirements and performance  measures. Specifically, we discuss both agencies’ efforts to ensure the  consistency and quality of these services, our recommendations to  improve these services, FAA’s recent efforts to establish requirements and  performance measures, and NWS’s plans for responding to these  requirements.
The information in this statement is based largely on our work supporting  the report being released today. In addition, to provide an update on the  agencies’ recent efforts, we reviewed key documents completed in  December 2007, including a new interagency agreement, FAA’s  requirements, and the accompanying quality assurance plan. We compared  NWS’s tentative next steps with best practices for validating requirements  and interviewed the NWS official responsible for responding to the new  requirements. We conducted our work on the report and testimony  between May 2007 and February 2008, in accordance with generally  accepted government auditing standards. Those standards require that we  plan and perform the audit to obtain sufficient, appropriate evidence to  provide a reasonable basis for our findings and conclusions based on our  audit objectives. We believe that the evidence obtained provides a  reasonable basis for our findings and conclusions based on our audit  objectives. Additional details on our objectives, scope, and methodology  are provided in appendix I.
subsections: section_title: Background
paragraphs: FAA is responsible for ensuring safe, orderly, and efficient air travel in the  national airspace system. NWS supports FAA by providing aviation-related  forecasts and warnings at air traffic facilities across the country. Among  other support and services, NWS provides four meteorologists at each of  FAA’s 21 en route centers to provide on-site aviation weather services.  This arrangement is defined and funded under an interagency agreement.
subsections: section_title: FAA’s Mission and Organizational Structure
paragraphs: FAA’s primary mission is to ensure safe, orderly, and efficient air travel in  the national airspace system. The agency’s ability to fulfill its mission  depends on the adequacy and reliability of its air traffic control systems, as  well as weather forecasts made available by NWS and automated systems.  These resources reside at, or are associated with, several types of  facilities: air traffic control towers, terminal radar approach control  facilities, air route traffic control centers (en route centers), and the Air  Traffic Control System Command Center. The number and functions of  these facilities are as follows:    517 air traffic control towers manage and control the airspace within  about 5 miles of an airport. They control departures and landings, as  well as ground operations on airport taxiways and runways.    170 terminal radar approach control facilities provide air traffic control  services for airspace within approximately 40 miles of an airport and  generally up to 10,000 feet above the airport, where en route centers’  control begins. Terminal controllers establish and maintain the  sequence and separation of aircraft.    21 en route centers control planes over the United States—in transit  and during approaches to some airports. Each center handles a  different region of airspace. En route centers operate the computer  suite that processes radar surveillance and flight planning data,  reformats them for presentation purposes, and sends them to display  equipment that is used by controllers to track aircraft. The centers  control the switching of voice communications between aircraft and  the center, as well as between the center and other air traffic control  facilities. Two en route centers also control air traffic over the oceans.
The Air Traffic Control System Command Center manages the flow of  air traffic within the United States. This facility regulates air traffic  when weather, equipment, runway closures, or other conditions place  stress on the national airspace system. In these instances, traffic  management specialists at the command center take action to modify  traffic demands in order to keep traffic within system capacity.
See figure 1 for a visual summary of the facilities that control and manage  air traffic over the United States.
section_title: NWS’s Mission and Organizational Structure
paragraphs: The mission of NWS—an agency within the Department of Commerce’s  NOAA—is to provide weather, water, and climate forecasts and warnings  for the United States, its territories, and its adjacent waters and oceans to  protect life and property and to enhance the national economy. In  addition, NWS is the official source of aviation- and marine-related  weather forecasts and warnings, as well as warnings about life-threatening  weather situations.
The coordinated activities of weather facilities throughout the United  States allow NWS to deliver a broad spectrum of climate, weather, water,  and space weather services in support of its mission. These facilities  include 122 weather forecast offices located across the country that  provide a wide variety of weather, water, and climate services for their  local county warning areas, including advisories, warnings, and forecasts;  9 national prediction centers that provide nationwide computer modeling  to all NWS field offices; and 21 center weather service units that are  located at FAA en route centers across the nation and provide  meteorological support to air traffic controllers.
section_title: NWS Provides Aviation Weather Services to FAA
paragraphs: As an official source of aviation weather forecasts and warnings, several  NWS facilities provide aviation weather products and services to the FAA  and aviation sector. These facilities include the aviation weather center,  weather forecast offices located across the country, and center weather  service units located at FAA en route centers. See table 1.
section_title: Center Weather Service Units: An Overview of Current Requirements
paragraphs: FAA’s existing requirements for the center weather service units are  broadly outlined in an interagency agreement that is updated every few  years. The interagency agreement specifies that NWS is to provide  meteorological advice and consultation to en route center operations  personnel and other designated FAA air traffic facilities within the en  route area of responsibility. This agreement establishes specific terms that  govern the number of NWS staff, their working hours, and cost  reimbursement details. It does not specify the contents, quality, or  frequency of weather products.
An NWS directive, signed in May 2006 and intended for NWS’s weather  forecast offices and center weather service units, provides more specific  information regarding the content of weather products and services,  including center weather advisories, daily briefings, on-demand  consultations, and meteorological impact statements. These products and  services are described in table 2. In addition, center weather service unit  meteorologists can provide input every 2 hours to the Aviation Weather  Center’s creation of the Collaborative Convective Forecast Product; train  FAA personnel on how to interpret weather information; and, if warranted,  provide weather briefings to nearby terminal radar approach control  facilities.
section_title: FAA Sought to Improve Aviation Weather Services Provided at En Route Centers
paragraphs: In recent years, FAA has sought to assess and improve the performance of  the center weather service units. For example, FAA performed multiple  studies on the current services provided by the center weather service  units that noted the lack of standardization of products and services. In  addition, FAA conducted a study to determine if remote operations were  feasible, and requested that NWS restructure its aviation weather services  to provide improved services more efficiently. In response to this request,  NWS conducted a prototype of remote operations in which center weather  service unit products and services were prepared by the closest weather  forecast office. NWS proposed expanding this prototype to FAA, but the  agency declined this proposal. Instead, FAA stated that it would redefine  its requirements for the functions provided by center weather service  units. Table 3 provides more information about the agency’s assessment  and improvement efforts.
section_title: FAA Found Its Requirements Were Not Sufficiently Precise and Worked to Develop New Ones
paragraphs: When FAA declined NWS’s proposal for restructured aviation weather services,  it did so in part because it considered its existing requirements governing  NWS’s center weather service units to be too broad to ensure the efficiency and  cost-effectiveness of the services. FAA then worked for several months to  redefine these requirements. In April 2007, FAA’s Air Traffic Organization began  refining its requirements for aviation weather services at the en route centers.  To do this, FAA collected relevant NWS and FAA orders and directives and  developed a list of over 100 products and services that the different service  units provide. FAA then sent this list to traffic managers in each of the en route  centers, asking them to specify the products and services that they need, the  ones they do not need, and any new products or services that they would like.  Traffic managers were also able to determine if they would want some of the  more customized weather products that are currently available at selected en  route centers. Using results from this survey, FAA developed a revised list of  requirements and performance measures, which it provided to NWS in late  December 2007.
section_title: Neither NWS nor FAA Currently Ensures the Quality of Aviation Weather Services at En Route Centers
paragraphs: While interagency agreements between NWS and FAA state that both  agencies have responsibilities for assuring and controlling the quality of  aviation weather observations, neither NWS nor FAA consistently does so  for weather products produced at the en route centers. Leading  organizations use quality assurance to provide staff and management with  objective insights into processes and associated work products.  Generally, quality assurance includes objectively evaluating performed  processes, work products, and services against applicable process  descriptions, standards, and procedures; identifying and documenting  noncompliance issues; providing feedback to project staff and managers  on the results of quality assurance activities; and ensuring that  noncompliance issues are addressed. However, neither NWS nor FAA has  developed and implemented performance measures and metrics, regularly  evaluated weather service unit performance, or provided feedback to  improve these aviation weather products and services.
Because of this lack of performance tracking and oversight, NWS cannot  demonstrate the quality or value of its services, and FAA cannot ensure  the value of the services it funds. As a result, it is not clear that FAA is  getting the information it needs to effectively manage air travel. FAA  officials stated that they intend to establish performance measures for  their redefined requirements and to improve their oversight against these  measures. However, FAA has not worked with NWS to define a  comprehensive set of measures for its requirements, and it is unclear how  the agency would develop a performance baseline for comparison to  actual performance because many of the products and services have not  previously been measured.
subsections: section_title: NWS Does Not Measure or Evaluate Aviation Weather Products and Services at En Route Centers
paragraphs: NWS does not measure or evaluate the aviation weather services it  provides at en route centers. Under existing interagency agreements, NWS  is responsible for controlling the quality of its aviation weather  observations. Specifically, the agency is responsible for monitoring and  evaluating the quality and effectiveness of its aviation weather services,  including the services provided at the weather forecast offices, the  Aviation Weather Center, and the en route centers.
While NWS has developed and continues to monitor performance  measures for aviation weather forecasts provided by its weather forecast  offices and the Aviation Weather Center, the agency has not done so for  the weather products and services provided at the en route centers.  Specifically, NWS has not developed performance measures for aviation  weather products and services at en route centers, evaluated the aviation  weather products and services developed at the en route centers, or  provided feedback for those services. NOAA and NWS officials declined to  explain why the agency does not have performance measures for aviation  weather products or services at en route centers, but they noted that  neither FAA nor NWS has required or funded such an effort. Further, the  aviation services branch chief told us that he had planned to begin  evaluations for weather unit services at the en route centers but decided to  wait because of the potential for large-scale changes to the services.
Until NWS establishes performance measures and evaluates the quality  and effectiveness of its products against these measures, the agency will  remain unable to ensure that it provides consistent quality products and to  effectively demonstrate the value it provides to FAA.
section_title: FAA Does Not Consistently Evaluate or Provide Feedback on Aviation Weather Services at En Route Centers
paragraphs: FAA has not consistently evaluated NWS services at its en route centers or  adequately provided feedback on the results of its few evaluations. Under  interagency agreements, FAA is responsible for ensuring that aviation  weather services meet its requirements. In addition, it requires the traffic  management officer within each traffic management unit to evaluate the  aviation weather services at the en route centers annually and to provide  feedback to the resident meteorologist-in-charge.
FAA has not consistently ensured the quality of aviation weather services  at en route centers. Specifically, it currently does not have any quantitative  and objective performance measures—such as timeliness, accuracy, or  false alarm rate—by which to evaluate these services. Agreements  between the agencies broadly specify the types of aviation weather  products to be developed at the en route centers but do not provide  criteria by which these products can be evaluated. In addition, FAA has  not consistently performed its annual evaluations of these products and  services. According to the contracting officer’s technical representative  responsible for the evaluations, the last evaluation was performed in 2006,  and its results were largely anecdotal. Specifically, the evaluation called  for the traffic management officer to rate the weather unit on a scale of 0  to 4 in different categories, including quality and timeliness of products  and services, knowledge of air traffic control, and participation in training.
The technical representative could not find any evaluations in 2005,  evaluations of only three service units in 2004, and evaluations of a  similarly small number of service units in 2003.
Further, FAA is not consistently providing feedback to weather staff at the  en route centers. According to the technical representative, the  evaluations from 2006 were not compiled or analyzed because the  evaluations contained no glaring problems or issues that needed additional  attention. In addition, the NWS aviation services chief told us that FAA  had sent him copies of the evaluations from 2006 but did not offer analysis  of these evaluations, express concerns about the services, or send the  results to the individual center weather service units. This official also  stated that he was not aware that FAA had performed any annual  evaluations of the center weather service units prior to 2006.
Because FAA has not established performance requirements or  consistently and thoroughly evaluated the aviation weather services at en  route centers, the agency cannot be sure that the products and services  provided by the weather unit meteorologists are adding value, and it  cannot provide feedback to NWS meteorologists in order to improve the  services. To address this shortfall, FAA officials stated that they intend to  establish performance measures for aviation weather services at en route  centers when they revise their requirements and to improve their oversight  of NWS against these measures. However, FAA has not worked with NWS  to develop measures for the products and services it will require from  NWS, and it is unclear how the agency would develop a performance  baseline for comparison to actual performance because many of the  products and services have not previously been measured.
section_title: Implementation of GAO Recommendations Should Help Address Performance Measurement Shortfalls
paragraphs: While many steps remain in defining the future of aviation weather  services at en route centers—including negotiations between FAA and  NWS on the provision of these services and FAA’s subsequent decision on  whether to obtain selected services from alternative sources—there are  steps both agencies can take to ensure that the quality of future aviation  weather products and services is measured and evaluated. In our  accompanying report released today,  we made two recommendations to  the Secretary of Commerce and three recommendations to the Secretary  of Transportation to improve the quality of aviation weather products and  services at en route centers.
We recommended that the Secretary of Commerce direct the Assistant  Administrator for the National Weather Service to    assist FAA in developing performance measures and metrics for the  products and services to be provided by center weather service units,  and    perform annual evaluations of aviation weather services provided at en  route centers and provide feedback to the center weather service units.
Further, we recommended that the Secretary of the Department of  Transportation direct the FAA Administrator to    work with NWS to define performance measures and metrics for  aviation weather services provided by meteorologists,    evaluate the services it receives against those measures and metrics,    ensure that results of these evaluations are provided to staff stationed  at center weather service units so that they can improve performance,  where applicable.
In written comments on a draft of our report, the Secretary of Commerce  agreed with our recommendations to assist FAA in developing  performance measures and metrics, and to perform annual evaluations of  aviation weather services and provide feedback to the center weather  service units. The department stated that after FAA provides its revised  requirements, it would work with FAA to develop methods for  performance monitoring and evaluation. Subsequently, on December 19,  2007, FAA provided its revised requirements to NWS.
The Department of Transportation’s Director of Audit Relations provided  comments on a draft of the report via e-mail. In those comments, the  department did not agree or disagree with our recommendations. The  department stated that FAA’s revised requirements are consistent with our  recommendations in that they establish performance measures and  evaluation procedures, and that FAA would begin to negotiate with NWS  to implement them. In addition, in late December 2007, after reviewing our  draft report, FAA and NWS signed a new interagency agreement that  requires FAA to develop performance standards and measures for the  assessment of center weather service units, and requires NWS to develop  and track metrics to support FAA’s performance measures.
section_title: FAA Identified New Aviation Weather Requirements and Performance Measures
paragraphs: FAA has already begun to address our recommendations; specifically, in  late December 2007, FAA finalized its new requirements for the aviation  weather services to be provided by center weather service units, which  include proposed performance measures and methods for evaluation. In  its requirements, FAA provides NWS with its overall vision for aviation  weather services, revises existing requirements, and defines a requirement  for a new product for terminal radar approach control facilities. In  addition, FAA identifies performance measures and processes for  evaluating performance and providing feedback to NWS.
FAA envisions services that are performance-based, standardized,  continuous, and have a national scope. Specifically, FAA requires  performance-based services that are measurable and allow for identifying  both successful performance and any performance problems. In addition,  FAA requests that the center weather service units provide standardized  services to all en route centers and increase their service coverage from 16  hours a day to 24 hours a day. Finally, FAA calls for transitioning the  scope of the center weather service units to monitor the entire national  airspace system, rather than the respective en route center regions. This  national scope is expected to allow more integrated decision making at the  national level while continuing to provide specialized products at the  regional and local levels.
In its new requirements, FAA also reiterates its need for existing products  and services and provides revisions to some of these requirements.  Specifically, FAA continues to require products such as twice-daily  briefings, center weather advisories, and the Collaborative Convective  Forecast Product. In addition, center weather service units will continue  to provide forecast coordination with other NWS offices, on-demand  advice and consultation, emergency planning, training, and dissemination  of a number of weather advisories into both NWS and FAA systems. Daily  briefings, however, will now be recorded, verified, and disseminated to  other facilities that do not receive an in-person briefing. In addition, on- demand consultation will be provided to en route centers, terminal radar  approach control facilities, towers, and the Air Traffic System Command  Center. According to the aviation services branch chief, this consultation  is currently provided only to en route centers, selected terminal radar  approach control facilities, and a small number of towers.
Further, the revised requirements define a new product: the terminal radar  approach control forecast product. This product is based in part on  decision aids currently used in select center weather service units, and on  requirements developed by a team consisting of aviation meteorological  stakeholders from industry and FAA. This forecast, which describes the  next 6 hours and is updated at least every 2 hours, will be presented in a  graphical format and include convection, winds, ceilings, and visibilities  for the area around terminal radar approach control facilities. FAA also  expects this product to include methods for verification and the  systematic collection of user feedback.
In addition to these requirements, FAA identifies performance measures as  well as processes for evaluating performance and providing feedback to  the forecasters. These performance measures include customer  satisfaction, forecast accuracy, and the aggregate of aircraft incidents  attributed to inaccurate aviation weather forecasts. Baselines for all of  these measures have not yet been developed. According to the chief of the  aviation services branch, NWS will propose additional performance  measures and develop baselines as it is able. To measure against these  performance measures, FAA has identified methods by which to evaluate  NWS. For example, to determine customer satisfaction, FAA plans to  develop a questionnaire for traffic management unit staff to be filled out  quarterly; to determine the aggregate of aircraft incidents attributed to  inaccurate aviation weather forecasts, FAA is to use safety statistics  currently tracked by the Air Traffic Organization. In addition, FAA is  planning to draw on NWS’s subject matter expertise to record and analyze  information to determine the accuracy of forecasts.
FAA expects NWS to respond as to whether it is able to meet the  requirements by early May 2008. In addition, FAA directed NWS to include  plans for three operational concepts (including technical and cost  information) for fulfilling the requirements—in its existing configuration  located at the 21 en route centers, through remote services provided by a  reduced number of regional facilities, and through remote services  provided by a single centralized facility. According to the requirements,  NWS’s response should assume a transition time of 90 days for the existing  configuration, 180 days for regionalized remote services, and 1 year for a  single facility. For each of these concepts, FAA has requested that NWS  include a technical plan, a facilities plan, a quality management plan, and a  plan for transitioning to the new approach. In addition, FAA asked that  NWS include cost plans for each of the concepts with the assumption of 1  base year and four 1-year options thereafter. The cost plan is also  expected to include estimated annual cost savings over this 5-year period.  FAA plans to select one of these operational concepts.
section_title: NWS Plans to Respond to FAA’s Requirements, but Proposed Transition Time Frames May Be Overly Ambitious
paragraphs: NWS plans to respond to FAA by the May 2008 deadline, but FAA’s  estimated time frames for providing the revised services may be overly  ambitious. NWS plans to submit its proposals for the three operational  concepts—the existing configuration located at the 21 en route centers,  remote services provided by a reduced number of regional facilities, and  remote services provided by a single centralized facility. FAA directed  NWS to assume a transition time of 90 days for the existing configuration,  180 days for regionalized remote services, and 1 year for a single facility.  To respond, NWS management created a team to analyze the  requirements, gain clarification on the requirements from FAA, and  develop the response. The NWS aviation services branch chief reported  that the agency is on track to respond by May 2008.
However, FAA’s proposed time frames for transitioning to a new  operational concept may be overly ambitious because of the activities that  NWS should perform before any transition. In accordance with best  practices and the opinion of the National Transportation Safety Board,  NWS intends to validate that the organization can provide these new  requirements—through a prototype or similar demonstration—before  transitioning to a new approach. Leading organizations validate  requirements to determine what impact the intended operational  environment will have on the ability to satisfy the stakeholders’ needs,  expectations, constraints, and interfaces. As part of this validation,  organizations explore the adequacy and completeness of requirements by  developing prototypes or simulations and by obtaining feedback about  them from relevant stakeholders. Given the importance of accurate and  timely weather information in air traffic control, it will be important for  NWS to conduct a thorough evaluation before it transitions to a new  operational concept in order to ensure that there are no impacts on the  continuity of air traffic operations and no degradation of weather service.
In addition, NWS has agreed to negotiate with its employees’ union, the  National Weather Service Employees Organization, whenever  organizational changes could affect working conditions—unless the union  has sufficient predecisional involvement. NWS’s employees’ union has a  representative on the team that is responding to FAA’s requirements, so  that later negotiations may be unnecessary. However, it is too soon to  determine whether negotiations will be needed and how long they will  take.
NWS’s aviation services branch chief agreed that FAA’s transition time  lines would be challenging. This official estimated that it could take over 2  years to transition to a new operational concept. To address this  disconnect between NWS’s capabilities and FAA’s expectations, NWS  plans to propose more feasible time frames in its response to FAA. FAA  officials reported that the agency will be open to NWS’s proposal.
In summary, even though center weather service units have been in  operation for over two decades, neither FAA nor NWS has implemented  performance measures and metrics, regularly evaluated weather service  unit performance, or provided feedback to improve these aviation weather  products and services. Until the agencies establish a system of  performance tracking and oversight, NWS will not be able to demonstrate  the quality or value of its services, and FAA will not be able to ensure the  value of the services it funds. To address these shortfalls, FAA has defined  more precise requirements that include performance measures and  evaluation methods, and NWS is working to respond to these  requirements. In its response, NWS is expected to prepare plans for three  alternative operational concepts for fulfilling these requirements. FAA will  choose one of the concepts. However, FAA’s proposed time frames for  transitioning to a new operational concept may be overly ambitious. Given  the potential for major changes to services and structure, NWS may  require additional time to properly validate the requirements, plan for any  necessary operational transitions, and ensure that aviation weather  services are not degraded by any potential changes.
Moving forward, it is important that FAA and NWS work together to  ensure that NWS’s aviation weather services address requirements and  that effective performance measures and evaluation methods are  established. This collaboration will help both agencies ensure the quality  and consistency of these services, and ensure that FAA has the  information it needs to effectively manage air traffic.
Mr. Chairman, this concludes my statement. I would be happy to answer  any questions that you or members of the subcommittee may have at this  time.
If you have any questions on matters discussed in this testimony, please  contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Other key  contributors to this testimony include Colleen Phillips (Assistant  Director), Kate Agatone, Amos Tevelow, and Jessica Waselkow.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this statement were to summarize selected sections of  our report being released today, including National Weather Service  (NWS) and Federal Aviation Administration (FAA) efforts to ensure the  consistency and quality of aviation weather services at en route centers,  and our recommendations to improve these services. In addition, we were  asked to provide an update on FAA’s recent efforts to develop aviation  weather requirements and performance measures, and NWS’s plans for  responding to these requirements.
To summarize selected sections of our report, we relied on the work  supporting that report. That report contains a detailed overview of our  scope and methodology.
In addition, to provide an update on FAA’s recent efforts to establish  requirements, we reviewed the new interagency agreement signed by both  agencies in late December 2007; FAA’s requirements sent to NWS on  December 19, 2007; and the accompanying quality assurance plan. We also  interviewed NWS’s aviation services branch chief to clarify which of these  requirements were new and which were revisions of current requirements.
To determine NWS’s plans for responding to these requirements, we  reviewed the new interagency agreement, FAA’s requirements, and the  accompanying quality assurance plan. We also interviewed the aviation  services branch chief, who is serving in an oversight role for NWS’s  response to FAA. We compared NWS’s tentative next steps with best  practices for validating requirements from the Capability Maturity Model®  Integration for Development.
We performed our work on the report and testimony from May 2007 to  February 2008. All of the work on which this testimony is based was  performed in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.