20.1 

 

ASSESSING THE UTILITY OF 

AN AUTOMATED 0-1 h THUNDERSTORM PRODUCT 

TO AIR TRAFFIC MANAGERS 

 

Dennis M. Rodgers,* Gregory Pratt, Lynn Sherretz 

NOAA Research – Forecast Systems Laboratory, Boulder, Colorado 

Thomas Amis 

National Weather Service, Center Weather Service Unit 

Fort Worth, Texas 

Young S. Chun, James T. Frimel 

NOAA Research – Forecast Systems Laboratory, Boulder, Colorado 

In collaboration with the Cooperative Institute for Research in the Atmosphere (CIRA), 

Colorado State University, Fort Collins, Colorado 

 

*

Corresponding author address: Dennis M. Rodgers, 

NOAA Research - FSL, FS5, 325 Broadway, Boulder, CO 

80305-3328; email: Dennis.M.Rodgers@noaa.gov 

1. INTRODUCTION 

NOAA Forecast Systems Laboratory (FSL), in 

conjunction with the NWS Prototype Aviation 

Collaborative Effort (PACE), evaluated the utility of 

a tactical convective hazard product (TCHP) at the 

Fort Worth Air Route Traffic Control Center 

(ARTCC – operationally designated ZFW) during a 

portion of the 2003 warm season. The TCHP 

graphically depicts current hazardous 

thunderstorms and a 1-h extrapolation forecast of 

hazardous thunderstorm positions. This automated 

product, generated every five minutes, is intended 

to support tactical air traffic decision-making. 

Responding to requirements referenced below, our 

goal in creating the TCHP is to consolidate all 

relevant thunderstorm information into a single 

graphical product, thereby mitigating possible 

effects of multiple products that could contain 

redundant or contradictory information. The TCHP 

is designed to capitalize on development of 

advanced products from the FAA Aviation Weather 

Research Program (AWRP: AUA-430) and to 

optimize the use of conventional advisories. The 

version of TCHP that we evaluated consists of 

detection and forecast information contained in the 

National Convective Weather Forecast (NCWF) and 

Convective SIGMETs. FAA AWRP developed the 

NCWF, and NWS/Aviation Weather Center 

forecasters produce Convective SIGMETs. 

The TCHP evaluation was designed to create a 

functional definition of a graphical tactical 

thunderstorm product for use by air traffic 

managers. Recommendations for the content and 

presentation of a TCHP are refined and its utility to 

operations is affirmed through the process of 

familiarizing the users with the characteristics of the 

product, demonstrating the product in an 

operational setting, and evaluating the product 

through feedback from the ZFW Traffic 

Management Unit (TMU) participants. 

The sponsors of this project are: 

• FAA Air Traffic System Requirements 

(ARS-100) 

• FAA Aviation Weather Research Program 

(AUA-430) 

• FAA Southwest Region Headquarters 

• National Weather Service Southern Region 

Headquarters 

Participants in this evaluation represent a multiagency collaboration. The NWS Southern Region 

Headquarters PACE project provided computer 

hardware and a high-speed (DSL) Internet 

connection; FSL provided software development 

and meteorological support; ZFW management 

hosted the evaluation and endorsed the 

participation of TMU traffic managers and 

supervisors; and the ZFW Center Weather Service 

Unit (CWSU) provided an area for PACE activities. 
The common goals of PACE and of the FAA TMU 

Weather Needs Project allowed beneficial 

leveraging of resources. FSL Aviation Division’s 

participation is in response to requirements and 

funding by the FAA Aviation Weather Research 

Program (AUA-430). 

2. BACKGROUND 

2.1 Reference Documents 

Two documents have guided this work. FAA 1999 

documents the results of an in-depth user needs 

analysis of weather information used in tactical air 

traffic decision-making. FAA 2002 sets forth 

requirements (master, supporting, and interim) for 

tactical convective forecasts for FAA traffic 

managers. TCHP is an initial step toward 

addressing unmet or newly identified weather 

information requirements of the Traffic Management 

Unit. 

2.2 Requirements 

A TMU Weather Needs Core Working Group was 

formed to address the above documented weather 

information requirements, beginning with 

convection products. The core working group 

includes: 

• Kevin Browne, FAA ARS-100 

• Craig Goff, AvMet/FAA ARS-100 

• Lynn Sherretz, Chief, FSL Aviation 

Program Development Branch 

• Greg Pratt, Chief, FSL Aviation Systems 

Development and Deployment Branch 

• Dennis Rodgers, FSL Development 

Meteorologist 

• Thomas Amis, ZFW CWSU Meteorologistin-Charge (MIC); Chief, PACE Operations 

• Paul Witsaman, Regional Aviation 

Meteorologist, NWS Southern Region 

Headquarters 

• Kathleen Schlachter, Aviation 

Meteorologist, NWS Headquarters 

At a meeting at ZFW in October 2002, the Core 

Working Group determined that initial requirements 

for a TCHP shall include 5-min updated graphical 

depictions of: 

• C-SIGMET Nowcast 

• C-SIGMET 1-h forecast 

• C-SIGMET text 

• NCWF Detection 

• NCWF 1-h forecast 

• NCWF Tops and Movement 

Another NWS product, which was considered for 

inclusion in TCHP, is the Center Weather Advisory 

(CWA). The CWA, issued by CWSUs, is a weather 

warning for conditions meeting or approaching 

AIRMET, SIGMET, or Convective SIGMET criteria. 

CWAs may be issued for a number of adverse 

weather conditions, including thunderstorms, and 

contain free-form text. Therefore decoding is 

necessary to discriminate thunderstorm CWAs from 

the rest. The decision was made to postpone 

inclusion of the CWA until it can be produced in a 

standardized machine-readable format. 

2.3 Development Environment 

Development of the TCHP was preceded by 

establishing the PACE Facility within the ZFW 

CWSU. In order to perform exploratory 

development in a rapid prototyping environment, a 

meteorological workstation isolated from ZFW 

operational systems was necessary. The FSL 

visualization platform called FX-Connect (FXC) was 

installed at the CWSU with a high-speed Internet 

connection to servers at FSL. Descriptions of FXC 

and PACE can be found in FSL 2002 and Rodgers 

and Amis 2002, respectively. NWS 2002 describes 

the concept of operations of PACE at ZFW. 

The first software engineering requirement involved 

decoding and creating AWIPS graphics for the 

TCHP components. Next, those products were 

integrated into the FXC menus and datastream, and 

FXC was enhanced with aviation map selections 

and tailored for use in ZFW airspace. PACE's FXC 

display system could then be used to test and 

validate early versions of the product. 

The PACE exploratory development facility proved 

to be an effective test bed for the TCHP 

development. Before the product was made 

available to TMU traffic managers, the PACE Chief 

and CWSU forecasters were able to examine TCHP 

components in their preliminary state on FXC. 

Based on their initial feedback, and comments 

solicited from TMU supervisors, new requirements 

were defined to change the background and map 

overlay colors to be consistent with the Integrated 

Terminal Weather System (ITWS) display. A 

second requirement was to provide animation. 

These modifications were implemented before the 

beginning of the evaluation. By incorporating the 

initial feedback from PACE, we were able to 
increase the likelihood that the product would be 

acceptable to the TMU users. 

This iterative development/feedback process 

represents rapid prototyping in a spiral software 

development model. In the spiral development 

model, an initial build is presented, feedback is 

gathered, requirements are added or modified, and 

improvements are incorporated at phased intervals. 

The TCHP is an evolutionary prototype in this 

development model. An evolutionary prototype, by 

definition, is built in a quality manner, experienced 

by the users, and evolved to better meet needs. 

The process is repeated as needed. This type of 

prototyping is flexible in that modified or added 

requirements and technological improvements may 

be incorporated quickly and efficiently. 

The PACE FXC system provided powerful 

capabilities to the evaluation. For the first time 

anywhere on any display system, we had the ability 

to view all of the TCHP components together with 

animating NEXRAD mosaic images as well as 

observations and analyses from the NWS AWIPS 

datastream. This allowed the CWSU/PACE 

meteorologists to examine TCHP performance, and 

to verify comments made by the TMU users 

regarding movement and forecast information from 

the NCWF. These unique animated displays also 

revealed skill in the C-SIGMET 1-h forecast when 

subjectively compared to the observations from the 

NEXRAD mosaic and NCWF detection field 

animations. 

3. TCHP DESCRIPTION 

3.1 TCHP Web-based Viewer 

We made the TCHP available for viewing on a 

password-protected web site accessible via the 

Internet. This approach is cost effective and poses 

no impact to ZFW operational systems. Links are 

available on the home page to documentation, 

publications, and a project contact list. In the center 

of the home page, a link invokes the Tactical 

Convective Hazards Product Viewer. The TCHP 

Viewer initializes with a default display on the 

ARTCC scale, including State, ARTCC, and 

Dallas/Fort Worth (DFW) and Houston (IAH) 

Terminal Approach Control (TRACON) boundaries, 

aviation location identifiers, and sets of selectors for 

CONUS, Houston Center, or TRACON displays, as 

well as a selection of data and map overlays, and a 

selector for an animated view. 

3.2 TCHP Graphical Content 

The evaluated version of the TCHP graphical 

display contains the following components: 

• NCWF Detection Field 

• NCWF 1-h forecast 

• Hazard Tops and Movement 

• C-SIGMET Nowcast 

• C-SIGMET 1-h forecast 

• C-SIGMET Text 

• Impacted Jet Route Display 

Components of the TCHP may be displayed in any 

combination. The default display includes Hazard 

Detection, Tops & Movement, and NCWF 1-hr 

forecast (Fig. 1). The three selectable C-SIGMET 

overlays on the TCHP selector menu are graphics 

derived from the conventional text Convective 

SIGMET. The C-SIGMET Nowcast graphic is the 

area polygon, line, or point as issued in the 

operational text Convective SIGMET. The CSIGMET 1-h forecast, unique to this project, is a 

graphic produced on the NCWF 5-min cycle, 

depicting the extrapolated position of the CSIGMET polygon 1 h from the current clock time, 

time-matched to the current NCWF 1-h forecast. In 

other words, the C-SIGMET 1-h forecast positions 

are valid at 1 h after issue time and updated every 5 

min to be valid up to 1 h, 55 min after issue time. 

The latest C-SIGMET 1-h Fcst is valid at the same 

time as the latest NCWF 1-h forecast. The 

extrapolated position of the C-SIGMET 1-h forecast 

is based on the thunderstorm area movement given 

in the operational Convective SIGMET text. 

Figure 1. TCHP default display.
The Impacted Jet Route graphic (Fig. 2) is a proofof-concept exercise involving a modest investment 

of developer time to produce a simple, "quick-look" 

graphic indicating thunderstorm-impacted air space. 

In this simplified presentation of the TCHP 

information, current and forecasted impact to 

specific jet routes can be determined at a glance. A 

subset of high-use jet routes are depicted in one of 

three colors: Green jet route segments indicate no 

direct impact from NCWF detected or forecast 

thunderstorms; Red jet route segments indicate 

impact from NCWF Detection level 3 or greater 

touching the jet route segment; Yellow jet route 

segments indicate a NCWF forecast polygon 

touching the jet route. This display concept offers a 

"red light/green light" simplified graphic display of 

weather impacted airspace, and could be easily 

extended to highlight sectors, approach/departure 

corridors and gates, etc. in addition to jet routes. 

Other automated, gridded hazardous weather 

diagnostic and forecast products (for example, icing 

and turbulence) could also be integrated into this 

type of display. This experimental display was 

included with the TCHP for traffic manager 

comments. 

3.3 Display Interface 

The left side of the TCHP Viewer contains controls 

for user-selectable data and map overlays, and five 

“radio buttons” for the five geographic areas; 

CONUS, ZFW ARTCC, ZHU ARTCC, DFW 

TRACON, IAH TRACON. Above the scale selector 

buttons, two sets of check boxes allow the user to 

select/deselect data and map overlays. A link 

below the scale selector buttons initializes an 

animation of the TCHP. The animation 

automatically loads and refreshes 6 detection 

images at 15-min intervals, providing 90 min of 

history. The seventh frame shows the most recent 

Detection with the most recent NCWF 1-h forecast 

and Tops & Movement overlays. 

The web-based TCHP viewer enabled TMU staff to 

experience the TCHP concept and content, but was 

not intended to be a comprehensive display system 

including all necessary operational functionality, 

such as extensive map overlay selections and 

aircraft situation information. The TMU participants 

frequently called those capabilities requirements. 

Nevertheless, the web display was an adequate 

cost-effective way to gather necessary feedback on 

the thunderstorm information content and 

presentation of the TCHP. 

Fig. 2. Impacted Jet Route Product. 

4. EVALUATION DESCRIPTION 

4.1 Training 

Nearly all traffic managers, traffic management 

coordinators and the CWSU meteorologists have 

completed training to become familiar with the use 

and interpretation of the thunderstorm products 

included in the TCHP. Additionally, participants 

were briefed on the evaluation plan and trained in 

the use of the project’s TMU web site to view the 

TCHP and associated displays. Once they became 

familiar with the components of the TCHP, TMU 

and CWSU participants were asked to provide 

feedback on the utility of the TCHP and suggestions 

for improvements. 

Training of TMU traffic managers began on 04 June 

and was completed on 27 June, with a total of 23 

participants attending a PowerPoint presentation 

individually or in pairs. The training sessions 

averaged 45 min in duration. The CWSU MIC 

completed some initial training in February and 

March 2003 but that training was limited to two 

TMU supervisors to facilitate early feedback 

gathering. Further training at that point had limited 

success due to changes in the TMU staff, changes 

in the TCHP web design, and the routine demands 

of MIC management duties. 

4.2 Schedules and Location 

The TCHP evaluation is located at the Fort Worth, 

TX ARTCC (ZFW). This site was selected due to 

the high volume of air traffic, technological 
advances, thunderstorm climatology, and location 

with respect to NWS/FAA regional headquarters. 

The CWSU is located within the Display System 

Replacement (DSR) Control Room next to the 

Operations Manager and TMU. This design 

facilitates a rapid flow of air traffic and weather data 

for use in tactical and strategic planning. The 

ARTCC operations floor is divided into two main 

areas: the DSR control room containing radar and 

monitor positions for controllers; and the traffic 

management area, in which TMU controllers, 

supervisors, and operations managers oversee all 

facets of air traffic through ZFW's airspace. The 

CWSU operates as part of the traffic management 

team. 

As agreed on by the project participants, the 

evaluation was conducted Monday through Friday 

from 0500 to 2145 local time when significant 

convection occurred. This provided the greatest 

opportunity for full participation by the PACE chief, 

CWSU meteorologists, ZFW TMU personnel and 

FSL developers. 

4.3 TCHP displays in an Operational Setting 

In general, the TCHP was displayed in the TMU 

alongside the Convective Collaborative Forecast 

Product (CCFP) during Strategic Planning 

telephone conference calls in which strategic and 

tactical planning of aircraft movement through the 

National Airspace System (NAS) is conducted. 

TMU supervisors and coordinators viewed TCHP at 

the TMU supervisory position, which is the hub of 

information flow within the TMU. This is where 

coordination of all traffic flow from the en-route 

airspace into the Dallas/Fort Worth (DFW) 

TRACON and terminal airspace occurs. 

In the CWSU, TCHP was displayed at the PACE 

operations computer and desktop computers with 

Internet connectivity. Occasionally, TCHP was 

displayed via a large screen plasma monitor to the 

traffic management and control room supervisory 

staff during daily operational briefings. 

4.4 Data Collection and Analysis Method 

The following information was obtained through 

one-on-one interviews with TMU traffic managers 

and supervisors, and through comments and 

discussions that occurred during training. Each 

individual respondent was trained and had ample 

opportunity to view the TCHP in real time during 

several active thunderstorm periods in June. 

Fourteen TMU traffic managers or supervisors (out 

of the 23 trained) provided feedback used in this 

report, representing 60% of the ZFW TMU staff. 

The CWSU MIC recorded their responses on a 

web-based TCHP evaluation form. Responses 

recorded on the evaluation web site were compiled 

and tallied, and combined with user comments to 

support this report. 

5. FINDINGS AND RECOMMENDATIONS 

5.1 Findings 

Several significant findings resulted from this 

operational assessment. 

• Traffic managers unanimously endorse the 

TCHP concept and most approved of the 

TCHP graphical display. 

• The NCWF is subject to erroneous motion 

vectors and extrapolated storm positions 

and, therefore, was considered 

insufficiently reliable to use in air traffic 

management decisions. 

• Traffic managers do not use Convective 

SIGMETs. 

• C-SIGMET 1-h extrapolated forecast 

graphic was useful when combined with 

NCWF Detection. 

• A direct benefit of conducting this 

evaluation is risk reduction in the 

operational implementation of a TCHP. A 

secondary benefit is the opportunity to “fine 

tune” the content and presentation of a 

graphic product prior to national 

implementation. 

A conceptual approach to differentiating 

thunderstorm information into tactical and strategic 

categories is described in the documents cited in 

Sec 2.1. For this exercise, the TCHP provided an 

explicit 1-h forecast location of NCWF Level 3 or 

greater storms. Beyond this tactical time frame, 

strategic thunderstorm information is operationally 

provided (by the operational CCFP) as percentage 

ranges of area coverage with a categorical 

probability of storms meeting the minimum area 

coverage threshold. An assumption was made that 

these two categories of thunderstorm forecasts are 

best presented as separate products; otherwise 

users might be confused by the differing graphical 

presentations. Results from this assessment 

indicate that the TMU user group is capable of 

assimilating and applying thunderstorm information 

presented in both forms, and a number of 
participants suggested combining them. Several 

traffic managers commented that the CCFP is used 

heavily for transcontinental overflight traffic 

decision-making, and requested that the time frame 

of the TCHP be extended by including the CCFP as 

a selectable overlay on the display, providing 0–6 h 

of thunderstorm information on one display. 

Our approach is to create one graphical display, 

which merged all relevant, operational thunderstorm 

information in the tactical time and space scales. 

The challenge is to present complex, detailed 

information in simple, easy-to-digest, stand-alone 

graphics that may then be integrated into decision 

support systems. We feel that the Impacted Jet 

Route product is a step in this direction. However, 

the responses from traffic managers in this TCHP 

evaluation demonstrate the need to provide as 

much flexibility as possible to the users to view 

tactical thunderstorm information within the context 

of their familiar display systems, complete with all of 

the user interactivity, map overlays, and aircraft 

targets routinely used by traffic managers. 

5.2 Recommendation for a Tactical Convective 

Hazard Product 

Based on the unanimous positive response of our 

sample group of ZFW TMU traffic managers and 

supervisors, implementation of a graphical tactical 

thunderstorm product is recommended, but 

additional prototyping is recommended to ensure 

maximum utility. The findings of this assessment 

indicate that this tactical product should include the 

following characteristics or attributes: 

• 0-2 h time frame for tactical applications 

• NCWF detection field with existing intensity 

levels and colors 

• Enhanced NCWF forecast graphic (1 h 

initially; 2 h when approved by Aviation 

Weather Technology Transfer Board) 

• Tops and Movement graphic 

• Selectable NCWF Performance graphic 

• Selectable Convective SIGMET 1-h 

forecast graphic showing extrapolated 

positions time-matched to the NCWF 

production cycle. 

• Selectable CCFP 2-, 4-, and 6-h graphics 

• Selectable sector and jet route map 

backgrounds (similar to ETMS or ITWS 

capabilities) 

• Selectable aircraft locations graphic 

• Typical user interactivity, e.g., zoom, pan, 

animate 

• Graphic presentation that is visually 

consistent with ITWS displays (neutral gray 

background; consistent map overlay 

selections and colors; etc.) 

The evaluation team suggests including a 

performance graphic as is found on the ADDS 

(http://adds.aviationweather.gov/) javascript 

presentation of NCWF. By including a performance 

graphic with the TCHP and providing the capability 

to overlay a time-matched extrapolated C-SIGMET 

forecast, uncertainties or gaps in forecast 

information with the automated product may be 

“filled in” with the less frequent human product, or 

flagged by the performance graphic. The 

performance graphic originally included in the 

NCWF graphics package provides a simple, visual 

method for quickly assessing the "goodness" of the 

NCWF in a given situation. This strategy 

maximizes the utility of both the conventional 

human-generated advisories and the frequently 

updated automated product. 

TMU respondents to the evaluation questionnaire 

indicated that the TCHP is most useful on the 

ARTCC scale, including a 150 nautical mile buffer 

outside of the ARTCC airspace. A conterminous 

US graphic would be useful with zoom and roam 

capability, applied to transcontinental overflight 

traffic decision-making. Most felt that ITWS is a 

superior display for gate and terminal area decisionmaking. 

The evaluation team recommends that work 

continue to improve the reliability of the storm 

motion and forecast position information provided 

by NCWF or subsequent versions of NCWF. 

Due to the number of comments regarding NCWF 

forecast reliability, the team recommends that the 

next phase of the evaluation include the enhanced 

version of the NCWF, as well as exploration of the 

utility of including the CCFP. 

Since the NCWF Detection is a hybrid product 

containing derived radar information and lightning, 

and differs in appearance and information content 

from a conventional radar reflectivity display, some 

training for users is strongly indicated. 

5.3 General Recommendations 

Based on the findings of this assessment, the 

evaluation team recommends that all future 

automated forecast products include real-time 

performance information as part of the operational 
graphics package as a requirement for AWTT Level 

D5 approval. A performance graphic, such as is 

found on the ADDS javascript presentation of 

NCWF, or a similar real-time verification graphic, 

facilitates the intelligent application of, and 

confidence in, the product by end users. 

Based on the limited positive responses from users, 

refinement of the Impacted Jet Route product 

according to the feedback obtained, and repeated 

evaluation of its utility in tactical decision making 

are justified. 

6. ACKNOWLEGEMENTS 

This work is in response to requirements and 

funding by the Federal Aviation Administration. The 

views expressed are those of the authors and do 

not necessarily represent the official policy or 

position of the FAA or NWS. 

7. REFERENCES 

FAA, 1999: Decision-Based Weather Needs for the 

Air Route Traffic Control Center (ARTCC) TMU. 

Federal Aviation Administration, Air Traffic Systems 

Requirements, ARS-100, Washington, D.C. 

November 1999, 21 pp. 

FAA, 2002: Weather Forecast Requirements in 

Support of the En Route Traffic Management Unit; 

Convection Products, Version 1.0. Federal Aviation 

Administration, Air Traffic Systems Requirements, 

ARS-100, Washington, D.C., July 2002, 25 pp 

FSL, 2002: FXC Users' Guide. NOAA Research - 

Forecast Systems Laboratory, 325 Broadway, 

Boulder CO 80305. [Available online at: 

http://wwwsdd.fsl.noaa.gov/FXC_UG/FXC_UG_TC.html.] 

NWS, 2002: Prototyping Aviation Collaboration 

Effort (PACE) concept of operations at the Fort 

Worth ARTCC CWSU. National Weather Service 

Southern Region Headquarters, Fort Worth, TX, 22 

pp. 

Rodgers, D.M., and T. Amis, 2002: Applying FXConnect to the Prototype Aviation Collaboration 

Effort at the Ft. Worth Air Route Traffic Control 

Center. Preprints, 10th Conf on Aviation, Range, 

and Aerospace Meteorology, Portland, OR, Amer. 

Meteor. Soc, Boston, MA, pp 277-280.