Assessing the Utility of an Automated 

0-1 h Tactical Convective Hazard Product 

to FAA Air Traffic Managers 

Interim Report Submitted to 

FAA ARS-100 and AUA-430 

By 

 

NOAA Forecast Systems Laboratory 

Boulder, Colorado 

In collaboration with 

NWS Prototyping Aviation Collaborative Effort (PACE) 

Ft Worth, Texas 

November 2003 
2

Summary 

Background. In response to FAA requirements for thunderstorm forecasts for traffic 

managers, NOAA Forecast Systems Laboratory (FSL) and the Ft Worth ARTCC/CWSU 

recently evaluated the utility of an automated graphical 0-1 hour thunderstorm forecast. 

The requirements call for a graphical tactical (0-2 hour) forecast and a graphical strategic 

(2-6 hour) forecast--each of which combine attributes of existing forecasts in order to 

mitigate negative impacts that can result from multiple forecasts that provide conflicting 

information. 

Our initial work focused on defining, creating, and evaluating a 0-1 hour graphical 

forecast (which we call the Tactical Convective Hazard Product; TCHP) that combined 

key attributes of two operational products: National Convective Weather Forecast 

(NCWF) and Convective SIGMETs. We chose to focus only on 0-1 hours because 

NCWF forecasts beyond one hour are not yet available. NCWF is generated 

automatically every five minutes and includes a detection field (based on radar and 

lightning observations) and 1 hour forecast based on extrapolation. Convective 

SIGMETs (produced hourly by NWS/Aviation Weather Center forecasters) depict 

current convection and include motion information from which we derived 1-hour 

forecasts via extrapolation. 

In addition to NCWF and Convective SIGMETs, the TCHP included: 1) map and data 

overlays (selectable by traffic managers); and 2) a jet route display with specific routes 

“colored” if currently impacted by convection or forecast to be impacted within 60 

minutes. 

Methods. The TCHP (which we made available via the public Internet from a passwordprotected web site at FSL) was displayed on the TMU supervisor’s computer. Using the 

Internet was cost-effective and avoided possible negative impacts on operational systems. 

TMU traffic managers were trained via a slide presentation and hands-on instruction. 

Feedback was gathered by the CWSU Meteorologist-in-Charge using structured and 

open-ended questions. 

Results Summary:

1) Traffic managers in our sample group unanimously endorsed the concept of the 

TCHP. Most reported that they were generally pleased with the “look-and-feel” 

of the TCHP and that it has potential to support operations. 

 

2) Most traffic managers reported that they do not use Convective SIGMETs and 

that its nowcast is not needed because NCWF provides detection information 

every 5 minutes. Some managers reported that the Convective SIGMET 1-hour 

forecast did have utility. 

3) Some traffic managers expressed concern about the reliability of the forecast 

component of the TCHP. We note that these responses are consistent with known 
3

strengths and weaknesses of the NCWF forecast. Erroneous motion vectors and 

the delay between detection of a hazardous storm and production of a forecast 

were cited as problems. We understand that the next version of NCWF (currently 

being readied for Aviation Weather Technology Transfer (AWTT) process level 

D3) has been enhanced to address these problems by incorporating output 

generated by the operational Rapid Update Cycle, radar trends, diurnal trends, and 

storm area coverage. Additional enhancements include improved motion vector 

calculations and more rigorous threshold of motions. 

4) Several traffic managers reported that the jet route display had potential if 

enhancements (e.g., capability to select specific routes) were made. 

5) Several traffic managers requested that we enhance the TCHP by enabling: 1) 

zooming and roaming; and 2) display of aircraft locations and the Collaborative 

Convective Forecast Product (CCFP), information currently available to traffic 

managers via the Enhanced Traffic Management System (ETMS). 

Conclusion and Recommendations: The evaluation showed that the TCHP has 

potential to support TMU operations--but is not ready for operational use. We 

recommend that the TCHP be re-evaluated after the following enhancements are made: 

• Replace the 1-hour forecast made by the current operational version of NCWF 

with the 1-hour forecast made by the next version of NCWF. 

• Remove the Convective SIGMET nowcast--but retain the Convective SIGMET 1-

hour forecast and train traffic managers on how to use it with NCWF. 

• Provide capability for zooming and roaming and enable the TCHP to display 

aircraft locations and CCFP. 

We also recommend that we explore including the NCWF 1-hour performance field in 

the TCHP. This field would enable traffic managers to assess how well NCWF 

performed in the previous hour. It could also make sense that CWSU meteorologists 

monitor the performance of NCWF and apprise traffic managers regarding its use. 
4

Table of Contents

 

 Page

Summary 2 

1. Introduction 5 

2. Background 6 

 2.1 Reference Documents 6 

 2.2 Requirements 6 

 2.3 Development Environment 7 

3. Tactical Convective Hazard Product Description 8 

 3.1 TCHP Web-based Viewer 8 

3.2 Graphical Content 8 

3.3 Display Interface 11 

4. Evaluation Description 12 

4.1 Training 12 

4.2 Schedule and Location 13 

4.3 TCHP Displays in an Operational Setting 14 

4.4 Challenges, Limitations, and Impacts to Project Schedule 14 

4.5 Data Collection and Analysis Method 15 

5. Evaluation Discussion 15 

5.1 Responses to Part A: TCHP Geographic Scales 15 

5.2 Responses to Part B: TCHP Graphical Components 15 

5.3 Responses to Part C: Utility of the TCHP to TMU Operations 16 

5.4 Responses to Part D: Impacted Jet Route Product 18 

6. Findings and Recommendations 19 

 6.1 Findings 19 

6.2 Recommendation for a Tactical Convective Hazard Product 20 

6.3 General Recommendations 21 

Appendix A: Responses to Evaluation Questionnaire 22 
5

1. INTRODUCTION 

NOAA Forecast Systems Laboratory (FSL), in conjunction with the NWS Prototyping 

Aviation Collaborative Effort (PACE), evaluated the utility of a tactical convective 

hazard product (TCHP) at the Fort Worth Air Route Traffic Control Center (ZFW) during 

a portion of the 2003 warm season. The TCHP graphically depicts current hazardous 

thunderstorms and a 1-h extrapolation forecast of hazardous thunderstorm positions. This 

automated product, generated every five minutes, is intended to support tactical air traffic 

decision-making. 

Responding to requirements referenced below, our goal in creating the TCHP is to 

consolidate all relevant thunderstorm information into a single graphical product, thereby 

mitigating possible effects of multiple products that could contain redundant or 

contradictory information. The TCHP is designed to capitalize on development of 

advanced products from the FAA Aviation Weather Research Program (AWRP: AUA430) and to optimize the use of conventional advisories. The version of TCHP that we 

evaluated consists of detection and forecast information contained in the National 

Convective Weather Forecast (NCWF) and Convective SIGMETs. FAA AWRP 

developed the NCWF and NWS/Aviation Weather Center forecasters produce 

Convective SIGMETs. 

The TCHP evaluation was designed to create a functional definition of a graphical 

tactical thunderstorm product for use by air traffic managers. Recommendations for the 

content and presentation of a TCHP are refined and its utility to operations is affirmed 

through the process of familiarizing the users with the characteristics of the product, 

demonstrating the product in an operational setting, and evaluating the product through 

feedback from the ZFW Traffic Management Unit (TMU) participants. 

The sponsors of this project are: 

• FAA Air Traffic System Requirements (ARS-100) 

• FAA Aviation Weather Research Program (AUA-430) 

• FAA Southwest Region Headquarters 

• National Weather Service Southern Region Headquarters 

Participants in this evaluation represent a multi-agency collaboration. The NWS 

Southern Region Headquarters PACE project provided computer hardware and a highspeed (DSL) Internet connection; FSL provided software development and 

meteorological support; ZFW management hosted the evaluation and endorsed the 

participation of TMU traffic managers and supervisors; and the ZFW Center Weather 

Service Unit (CWSU) provided an area for PACE activities. The common goals of PACE 

and of the FAA TMU Weather Needs Project allowed beneficial leveraging of resources. 

FSL Aviation Division’s participation is in response to requirements by FAA ARS-100 

and funding by FAA AUA-430. 
6

2. BACKGROUND 

2.1 Reference Documents 

Decision-Based Weather Needs for the Air Route Traffic Control Center (ARTCC) TMU

(FAA, Air Traffic System Requirements ARS-100, November 1999, 21 pp). This report 

documents the results of an in-depth user needs analysis of weather information used in 

tactical air traffic decision making. 

Weather Forecast Requirements in Support of the En Route Traffic Management Unit; 

Convection Products, Version 1.0 (FAA ARS-100, July 2002, 25 pp). This document sets 

forth requirements (master, supporting, and interim) for tactical convective forecasts for 

FAA traffic managers. TCHP is an initial step towards addressing unmet or newly 

identified weather information requirements of the Traffic Management Unit. 

2.2 Requirements 

A TMU Weather Needs Core Working Group was formed to address the above 

documented weather information requirements, beginning with convection products. The 

core working group includes: 

• Kevin Browne, FAA ARS-100 

• Craig Goff, AvMet/FAA ARS-100 

• Lynn Sherretz, Chief, FSL Aviation Program Development Branch 

• Greg Pratt, Chief, FSL Aviation Systems Development and Deployment Branch 

• Dennis Rodgers, FSL Development Meteorologist 

• Thomas Amis, ZFW CWSU Meteorologist-in-Charge (MIC); Chief, PACE 

Operations 

• Paul Witsaman, Regional Aviation Meteorologist, NWS Southern Region 

Headquarters 

• Kathleen Schlachter, Aviation Meteorologist, NWS Headquarters 

At a meeting at ZFW in October 2002, the Core Working Group determined that initial 

requirements for a TCHP shall include 5-min updated graphical depictions of: 

• C-SIGMET Nowcast 

• C-SIGMET 1-h Forecast 

• C-SIGMET text 

• NCWF Detection 

• NCWF 1-h Forecast 

• NCWF Tops and Movement 

Another NWS product, which was considered for inclusion in TCHP, is the Center 

Weather Advisory (CWA). The CWA, issued by CWSUs, is a weather warning for 

conditions meeting or approaching AIRMET, SIGMET, or Convective SIGMET criteria. 
7

CWAs may be issued for a number of adverse weather conditions, including 

thunderstorms, and contain free-form text. Therefore developing decoding capability is 

necessary to discriminate thunderstorm CWAs from the rest. The decision was made to 

postpone including the CWA until it can be produced in a standardized machine-readable 

format. 

2.3 Development Environment 

Development of the TCHP was preceded by establishing the PACE Facility within the 

ZFW CWSU. In order to perform exploratory development in a rapid prototyping 

environment, a meteorological workstation isolated from ZFW operational systems was 

necessary. The FSL visualization platform called FX-Connect (FXC) was installed at the 

CWSU with a high-speed Internet connection to servers at FSL. Using FXC is costeffective because FXC was developed by NOAA and uses the standard AWIPS database. 

Descriptions of FXC and PACE can be found in: 

• Rodgers, D.M., and T. Amis, 2002: Applying FX-Connect to the Prototyping 

Aviation Collaboration Effort at the Ft. Worth Air Route Traffic Control Center. 

Preprints, 10th Conf on Aviation, Range, and Aerospace Meteorology, Portland, 

OR, Amer. Meteor. Soc., Boston, MA, pp 277-280. 

• National Weather Service, 2002: Prototyping Aviation Collaboration Effort 

(PACE) concept of operations. NWS Southern Region Headquarters, Ft Worth, 

TX, 22 pp. 

The first software engineering requirement involved decoding and creating graphics for 

the TCHP components. Next, those graphics were integrated into the FXC menus and 

datastream, and FXC was enhanced with aviation maps and tailored for use in ZFW 

airspace. PACE's FXC display system could then be used to test and validate early 

versions of the product. 

The PACE exploratory development facility proved to be an effective test bed for the 

TCHP development. Before the product was made available to TMU traffic managers, 

the PACE Chief and CWSU forecasters were able to examine TCHP components in their 

preliminary state on FXC. Based on their initial feedback, and comments solicited from 

TMU supervisors, new requirements were defined to change the background and map 

overlay colors to be consistent with the Integrated Terminal Weather System (ITWS) 

display. A second requirement was to provide animation. These modifications were 

implemented before the beginning of the evaluation. By incorporating the initial 

feedback from PACE, we were able to increase the likelihood that the product would be 

acceptable to TMU users. 

This iterative development/feedback process represents rapid prototyping in a spiral 

software development model. In the spiral development model, an initial build is 

presented, feedback is gathered, requirements are added or modified, and improvements 

are incorporated at phased intervals. The TCHP is an evolutionary prototype in this 

development model. An evolutionary prototype, by definition, is built in a quality 
8

manner, experienced by the users, and evolved to better meet needs. The process is 

repeated as needed. This type of prototyping is flexible in that modified or added 

requirements and improvements in technology may be incorporated quickly and 

efficiently. 

The PACE FXC system provided powerful capabilities to the evaluation. For the first 

time anywhere on any display system, we had the ability to view all of the TCHP 

components together with animating NEXRAD mosaic images as well as observations 

and analyses from the NWS AWIPS datastream. This allowed the CWSU/PACE 

meteorologists to examine TCHP performance, and to verify comments made by the 

TMU users regarding movement and forecast information from the NCWF. These 

unique animated displays also revealed skill in the C-SIGMET 1-h forecast when 

subjectively compared to the observations from the NEXRAD mosaic and NCWF 

detection field animations. 

3. TCHP DESCRIPTION 

3.1 TCHP Web-based Viewer 

We made the TCHP available for viewing on a password-protected web site accessible 

via the Internet (Fig.1). This approach is cost effective and poses no impact to ZFW 

operational systems. Links are available on the home page to documentation, 

publications, and a project contact list. In the center of the home page, a link invokes the 

Tactical Convective Hazards Product Viewer (Fig. 2). The TCHP Viewer initializes with 

a default display on the ARTCC scale, including State, ARTCC, and Terminal Approach 

Control (TRACON) boundaries, aviation location identifiers, and sets of selectors for 

CONUS, Houston Center, or TRACON displays, as well as a selection of data and map 

overlays, and a selector for an animated view. 

3.2 TCHP Graphical Content 

The evaluated version of the TCHP graphical display contains the following components: 

• NCWF Detection Field 

• NCWF 1-h Forecast 

• Hazard Tops and Movement 

• C-SIGMET Nowcast 

• C-SIGMET 1-h Forecast 

• C-SIGMET Text 

• Impacted Jet Route Display 
9

Figure 1. TMU Project home page. 

Components of the TCHP may be displayed in any combination. The default display 

includes Hazard Detection, Tops & Movement, and NCWF 1-h Forecast (Fig. 2). The 

three selectable C-SIGMET overlays on the TCHP selector menu are graphics derived 

from the conventional text Convective SIGMET. The C-SIGMET Nowcast graphic is 

the area polygon, line, or point as issued in the operational text Convective SIGMET. 

The C-SIGMET 1-h Forecast, unique to this project, is a graphic produced on the NCWF 

5-min cycle, depicting the extrapolated position of the C-SIGMET polygon 1 h from the 

current clock time, time-matched to the current NCWF 1-h forecast. In other words, the 

C-SIGMET 1-h Forecast positions are valid at 1 h after issue time and updated every 5 

min to be valid up to 1 h 55 min after issue time. The latest C-SIGMET 1-h Fcst is valid 

at the same time as the latest NCWF 1-h forecast. The extrapolated position of the CSIGMET 1-h Forecast is based on the thunderstorm area movement given in the 

operational Convective SIGMET text. The C-SIGMET Text graphic overlay contains the 

observed and/or forecasted phenomena portion of the original text message. 

The Impacted Jet Route graphic (Fig. 3) is a proof-of-concept exercise involving a 

modest investment of developer time to produce a simple, "quick-look" graphic 

indicating thunderstorm-impacted air space. In this simplified presentation of the TCHP 
10

information, current and forecasted impact to specific jet routes can be determined at a 

glance. A subset of high-use jet routes are depicted in one of three colors: Green jet route 

segments indicate no direct impact from NCWF Detection or Forecast thunderstorms; 

Red jet route segments indicate impact from NCWF Detection level 3 or greater touching 

the jet route segment; Yellow jet route segments indicate a NCWF Forecast polygon 

touching the jet route. This display concept offers a "red light/green light" simplified 

graphic display of weather impacted airspace, and could be easily extended to highlight 

sectors, approach/departure corridors and gates, etc. in addition to jet routes. Other 

automated, gridded hazardous weather diagnostic and forecast products (for example, 

icing and turbulence) could also be integrated into this type of display. This experimental 

display was included with the TCHP for traffic manager comments. 

 

Figure 2. TCHP Viewer, default display condition. 
11

Figure 3. Impacted Jet Route display.

3.3 Display Interface 

The left side of the TCHP Viewer contains controls for user-selectable data and map 

overlays, and five “radio buttons” for the five geographic areas; CONUS, ZFW ARTCC, 

ZHU ARTCC, DFW TRACON, IAH TRACON (Fig. 4). Above the scale selector 

buttons, two sets of check boxes allow the user to select/deselect data and map overlays. 

A link below the scale selector buttons (not shown) initializes an animation of the TCHP. 

The animation automatically loads and refreshes 6 detection images at 15-min intervals, 

providing 90 min of history. The seventh frame shows the most recent Detection with the 

most recent NCWF 1-h Forecast and Tops & Movement overlays. 

The web-based TCHP viewer enabled TMU staff to experience the TCHP concept and 

content, but was not intended to be a comprehensive display system including all 

necessary operational functionality such as extensive map overlay selections and aircraft 

situation information. The TMU participants frequently called those capabilities 

requirements. Nevertheless, the web display was an adequate cost-effective way to 

gather necessary feedback on the thunderstorm information content and presentation of 

the TCHP. 
12

Figure 4. TCHP scale, map, and data overlay selectors. 

4. EVALUATION DESCRIPTION 

4.1 Training 

Nearly all traffic managers, traffic management coordinators, and the CWSU 

meteorologists have completed training to become familiar with the use and 

interpretation of the thunderstorm products included in the TCHP. Additionally, 

participants were briefed on the evaluation plan and trained in the use of the project’s 

TMU web site to view the TCHP and associated displays. Once they became familiar 

with the components of the TCHP, TMU and CWSU participants were asked to provide 

feedback on the utility of the TCHP and suggestions for improvements. 

Training of TMU traffic managers began on 04 June and was completed on 27 June, with 

a total of 23 participants attending a PowerPoint presentation individually or in pairs. 

This was later than our initial target because of challenges and limitations described in 

Section 4.4. The training sessions averaged 45 minutes in duration. The CWSU MIC 
13

completed some initial training in February and March 2003 but that training was limited 

to two TMU supervisors to facilitate early feedback gathering. Further training at that 

point had limited success due to changes in the TMU staff, changes in the TCHP web 

design, and the management duties of the MIC. 

Date TMU Specialist Trainee

6/4 Bob Weingarten 

6/4 Mary Hokit (Sup) 

6/5 John Sims 

6/5 Dave Collins 

6/6 Dennis McCain 

6/6 Richard Sherpy 

6/6 Kevin Davis 

6/10 Ron Blair 

6/10 Ken Woodham 

6/11 Mark Herriage 

6/11 Lynn Leverenz (Sup) 

6/11 Bruce Stephenson 

6/11 Hugh Hunton 

6/11 Mike Bradley 

6/11 Michelle Foster (Sup) 

6/26 Javier Morales 

6/26 Susan Conley 

6/26 Gean Paden 

6/26 Darren Harris 

6/26 Mike Clifton 

6/26 Al Guerra (Sup) 

6/27 Alex Dedominicis 

6/27 Danny Walters 

4.2 Schedule and Location 

The TCHP evaluation was conducted at the Fort Worth, TX ARTCC (ZFW). This site 

was selected due to the high volume of air traffic, thunderstorm climatology, and location 

with respect to NWS/FAA regional headquarters. The CWSU is located within the 

Display System Replacement (DSR) Control Room next to the Operations Manager and 

TMU. This design facilitates a rapid flow of air traffic and weather data for use in 

tactical and strategic planning. The ARTCC operations floor is divided into two main 

areas: the DSR control room containing radar and monitor positions for controllers; and 

the traffic management area, in which TMU controllers, supervisors, and operations 

managers oversee all facets of air traffic through the Center's airspace. The CWSU 

operates as part of the traffic management team. 

As agreed on by the project participants, the evaluation was conducted Monday through 

Friday from 0500 to 2145 local time when significant convection occurred. This 
14

provided the greatest opportunity for full participation by the PACE Chief, CWSU 

meteorologists, ZFW TMU personnel and FSL developers. 

4.3 TCHP Displays in an Operational Setting 

In general, the TCHP was displayed in the TMU alongside the Convective Collaborative 

Forecast Product (CCFP) during Strategic Planning Telcons in which strategic and 

tactical planning of aircraft movement through the National Airspace System (NAS) is 

conducted. TMU supervisors and coordinators viewed TCHP at the TMU supervisory 

position, which is the hub of information flow within the TMU. This is where 

coordination of all traffic flow from the en-route airspace into the Dallas/Fort Worth 

(DFW) TRACON and terminal airspace occurs. 

In the CWSU, TCHP was displayed at the PACE operations computer and desktop 

computers with Internet connectivity. Occasionally, TCHP was displayed via a large 

screen plasma monitor to the traffic management and control room supervisory staff 

during operational shift briefings. 

4.4 Challenges, Limitations, and Impacts to Project Schedule 

The project initially experienced a number of challenges and limitations before the TCHP 

assessment could begin. Due to congestion of the FAA Local Area Network we found 

that we had to change the TCHP display format from GIF images to PNG to achieve 

reasonable download times with animation. This compression of image files enabled 

TCHP to be displayed in a reasonable time that would not overburden the TMUs time 

when evaluating products. 

A major delay of the project start-up occurred due to an FAA National Facility 

Evaluation of the ZFW ARTCC. Since many resources were needed in preparing for this 

evaluation, it was decided by local FAA management not to engage in any new activities 

from March through early May, the time when most organized convective events affect 

ZFW airspace. 

Evaluating the TCHP in an operational environment was difficult at times due to the high 

tempo operations in the TMU during convective events when rapid decisions need to be 

made. Additionally, throughout the TCHP evaluation there was a heightened sensitivity 

to avoiding all unnecessary distractions to the controllers on duty. The result was limited 

accessibility to the TMU traffic managers by project personnel. Some TMU supervisors 

and coordinators were understandably resistant to using unproven technology in their 

decision making process. This justifiable reluctance is coupled with the general tendency 

of operational people to resist accepting new tools or technology to avoid disrupting their 

operational routine. 

Finally, NOAA servers were attacked by hackers on at least two occasions during the 

spin-up of this evaluation. This caused increased security measures to be enacted by all 
15

NOAA Labs, and resulted in limited access to the TCHP web site during the late winter 

and early spring. This situation delayed training and the beginning of the evaluation. 

4.5 Data Collection and Analysis Method

 

The following information was obtained through one-on-one interviews with TMU traffic 

managers and supervisors, and through comments and discussions that occurred during 

training. Each individual respondent was trained and had ample opportunity to view the 

TCHP in real time during several active thunderstorm periods in June. 

Fourteen TMU traffic managers or supervisors (out of the 23 trained) provided feedback 

used in this report, representing 60% of the ZFW TMU staff. The CWSU MIC recorded 

their responses on a web-based TCHP evaluation form. Responses recorded on the 

evaluation web site were compiled and tallied, and combined with user comments to 

support this report. The results from each of the evaluation questions are discussed 

below. The actual responses and comments from the TMU participants are reproduced in 

Appendix A. 

5. EVALUATION DISCUSSION 

5.1 Responses to Part A: TCHP Geographic Scales. 

In the three questions (A1-A3) regarding the geographic coverage of the available 

graphics, all 14 traffic managers found the ARTCC scale useful, easy to use, and 

readable, while a majority had negative response or no opinion on the utility of the 

CONUS and TRACON scales (10 and 8, respectively). Comments included suggestions 

that the CONUS and ARTCC products would be more useful if zoom and pan 

capabilities were added. A majority stated a preference of ITWS over the TCHP for the 

TRACON and Arrival Gate scale. 

5.2 Responses to Part B: TCHP Graphical Components 

Question B1 is about background map selections in the TCHP. All 14 of the traffic 

managers found the default maps to be adequate, though a majority (9) stated more 

optional choices were needed. Issues of map backgrounds, selectable jetroutes, zoom and 

pan, and aircraft situation data relate to the chosen operational platform rather than as 

attributes of a TCHP. 

A stand-alone TCHP display would require all of the display attributes mentioned in the 

user comments in Part B. However, all of these display attributes and selectable map 

overlay capabilities exist on FAA operational display systems into which a final form of 

TCHP may be integrated. 

The next set of questions dealt with the display characteristics of the TCHP components. 

All traffic managers found the NCWF Detection, Forecast, and Tops and Movement 
16

graphics satisfactory, and most (11) reported that the number of levels of information was 

adequate in the Detection product. However, when asked about Convective SIGMET 

Nowcast and Convective SIGMET Forecast graphic attributes, a large majority said CSIGMETs are rarely used for any operational decision-making. 

5.3 Responses to Part C: Utility of the TCHP to TMU Operations 

Responding to Question C1, the traffic managers were unanimous in their opinion that 

this type of display is useful to operations. Question C2, asking for ways to improve the 

TCHP, resulted in many requests for the map backgrounds, display controls, and aircraft 

traffic information found on their operational display systems, and few suggestions for 

the improvement of the thunderstorm product itself. Several commented that integration 

of the 2-hr CCFP would be useful for tactical decisions involving transcontinental 

overflight traffic. Interestingly, participants did not request that Center Weather 

Advisories (CWA) be included in the TCHP. A comment pertaining to the information 

content of the TCHP was to add thunderstorm growth and decay indicators (in response 

to question A2). Two responses calling for animation were from the first trainees who 

provided early feedback. Based in part on their recommendation, animation was 

implemented for the remainder of the evaluation. 

Question C3, asking if NCWF Detection influenced decision-making, could have been 

reworded to avoid the idea of using the TCHP for operational decisions; it is not an 

operational system, therefore could not be used for operational decisions. Otherwise, 

comments were generally positive. 

In responses to C4, the number of comments regarding "detail" compared to Nexrad may 

indicate a training issue. Apparently, some traffic managers misunderstood the 

distinction between the filtered NCWF Detection product and conventional base 

reflectivity displays, although this topic was thoroughly covered in each training session. 

Therefore, no conclusions can be drawn regarding agreement/disagreement between 

conventional radar displays and the NCWF Detection. 

In Question C5 regarding the utility of the NCWF, a majority of traffic managers (8) 

stated a reluctance to use the NCWF due to inconsistent or obviously erroneous motion 

vectors and forecast positions. The unreliability of the motion and forecast position is a 

significant concern to the users. Comments from the respondents included: 

• " NCWF was not reliable enough to make tactical decisions with, it either took 

too long for forecast to become available or the movement was inconsistent with 

other data sources such as ITWS and WARP." 

• " Noticed a 180 degree difference in movement this morning at 1131z in ZHU 

ARTCC airspace. This does not create confidence in the product." 

• "No, again this is not an operational system. Also, we noted that during this event 

the NCWF was 90 degrees off the actual movement of the cells. At 

approximately 2230z NCWF indicated movement from around 280 degrees at 19 
17

knots while the detected movement through looping and ITWS both indicated 

cells moving from around 020 degrees at 5 to 10 knots." 

• "NCWF is not reliable enough to make tactical air traffic decisions from yet." 

In comments from C6-C8, an overwhelming majority of traffic managers responded that 

the C-SIGMET is not used in TMU decision-making. However, a few commented that 

the C-SIGMET 1-Hr Fcst helped fill in movement information in the cases where NCWF 

failed to produce motion and forecast position information. Despite a negative 

predisposition towards the operational utility of Convective SIGMETS (see, for example, 

responses to B8, C6, and C7, Appendix A), several respondents recognized the utility of 

the extrapolated C-SIGMET forecast. 

Questions C7 and C8 ask if the C-SIGMET 1-h forecast has utility and how it compares 

to NCWF. Responses included: 

• "Yes, because it reconfirms movement..." 

• "Yes, because it gave me more information to make a decision." 

• "Somewhat, it gave more believability to the NCWF" 

• "It aided and filled in holes left by the NCWF" 

• "The convective sigmet forecast was more in line with actual movement of 

thunderstorms. NCWF was not always reliable therefore unusable in air traffic 

decision-making." 

• "Convective Sigmet was better in catching early movement. Too many areas do 

not display movement making it hard to base decision on NCWF." 

An example of the benefit of a C-SIGMET Forecast overlay is shown in Fig 5, in a 

situation where the NCWF did not produce a Tops and Movement or Forecast position 

for the slowly diminishing area of thunderstorms near the DFW terminal area. The CSIGMET 1-h forecast positions (64C area and 65C line) combined with the NCWF 

Detection give the user current coverage and intensity information and useful 1-h position 

information, after NCWF stopped producing movement and forecast information for 

elements of that system. 

Results for question C9, C10 were mixed, but positive. Some felt that the NCWF is not 

ready for operational use or that it is too early to have developed confidence in the 

product. A general sentiment is that with improvements in reliability, the TCHP will 

contribute to safer and more efficient operations. Several responded that "Tactical" 

should extend to two hours and include the CCFP. 

Responses to questions C12 and C13 indicated that training and documentation were 

adequate, with four compliments regarding training. 
18

Figure 5 C-SIGMET 1-hr forecast with NCWF Detection 

5.4 Responses to Part D: Impacted Jet Route Product 

The experimental Impacted Jet Route Product received limited encouraging feedback. 

According to the responses, this product might be useful as a “quick-look” product to 

identify thunderstorm-impacted jet routes after making the recommended changes 

including the addition of user selectable jet routes and thicker lines. It was generally felt 

that this product is not ready to be used operationally in its present form, and that it 

suffered from the identified NCWF reliability problems. 
19

6. FINDINGS AND RECOMMENDATIONS 

6.1 Findings 

Several significant findings resulted from this operational assessment. 

• Traffic managers unanimously endorse the TCHP concept and most approved of 

the TCHP graphical display (Questions A2, B1-B7, C1). 

• The NCWF is subject to erroneous motion vectors and extrapolated storm 

positions and, therefore, was considered insufficiently reliable to use in air traffic 

management decisions (Question C5). 

• Traffic managers do not use Convective SIGMETs (Questions C6, C7). 

• C-SIGMET 1-h extrapolated forecast graphic showed utility when combined with 

NCWF Detection (Question C8). 

• A direct benefit of conducting this evaluation is risk reduction in the operational 

implementation of a TCHP. A secondary benefit is the opportunity to “fine tune” 

the content and presentation of a graphic product prior to national 

implementation. 

A conceptual approach to differentiating thunderstorm information into tactical and 

strategic categories is described in the documents cited in Sec 2.1. For this exercise, the 

TCHP provided an explicit 1-h forecast location of NCWF Level 3 or greater storms. 

Beyond this tactical decision making time frame, strategic thunderstorm information is 

operationally provided (CCFP) as a percentage of area coverage with a categorical 

probability of storms meeting the minimum area coverage threshold. An assumption was 

made that these two categories of thunderstorm forecasts are best presented as separate 

products; otherwise users might be confused by the differing graphical presentations. 

Results from this assessment indicate that the TMU user group is capable of assimilating 

and applying thunderstorm information presented in both methods, and a number of 

participants suggested combining them. Several traffic managers commented that the 

CCFP is used heavily for transcontinental overflight traffic decision-making, and 

requested that the time frame of the TCHP be extended by including the CCFP as a 

selectable overlay on the display, providing 0 - 6 h of thunderstorm information on one 

display. 

Our approach is to create one graphical display, which merges all relevant, operational 

thunderstorm information in the tactical time and space scales. The challenge is to 

present complex, detailed information in simple, easy-to-digest, stand-alone graphics that 

may then be integrated into decision support systems. We believe that the Impacted Jet 

Route product is a step in this direction. However, the responses from traffic managers in 

this TCHP evaluation demonstrate the need to provide as much flexibility as possible to 

the users to view tactical thunderstorm information within the context of their familiar 

display systems, complete with all of the user interactivity, map overlays, and aircraft 

targets routinely used by traffic managers. 
20

6.2 Recommendation for a Tactical Convective Hazard Product 

Based on the unanimous positive response of our sample group of ZFW TMU traffic 

managers and supervisors to question C1, implementation of a graphical tactical 

thunderstorm product is recommended, but additional prototyping is recommended to 

ensure maximum utility. The findings of this assessment indicate that this tactical 

product should include the following characteristics or attributes: 

• 0-2 h time frame for tactical applications 

• NCWF detection field with existing intensity levels and colors 

• Enhanced NCWF Forecast graphic (1 h initially; 2 h when approved by Aviation 

Weather Technology Transfer Board) 

• Tops and Movement graphic 

• Selectable NCWF Performance graphic 

• Selectable Convective SIGMET 1-h forecast graphic showing extrapolated 

positions time-matched to the NCWF production cycle. 

• Selectable CCFP 2-, 4-, and 6-h graphics 

• Selectable sector and jet route map backgrounds (similar to ETMS or ITWS 

capabilities) 

• Selectable aircraft locations graphic 

• Typical user interactivity, e.g., zoom, pan, animate 

• Graphic presentation that is visually consistent with ITWS displays (neutral gray 

background; consistent map overlay selections and colors; etc.) 

The evaluation team suggests including a performance graphic as is found on the ADDS 

(http://adds.aviationweather.gov) javascript presentation of NCWF. By including a 

performance graphic with the TCHP and providing the capability to overlay a timematched extrapolated C-SIGMET forecast, uncertainties or gaps in forecast information 

with the automated product may be “filled in” with the less frequent human product, or 

flagged by the performance graphic. The performance graphic originally included in the 

NCWF graphics package provides a simple, visual method for quickly assessing the 

"goodness" of the NCWF in a given situation. This strategy maximizes the utility of both 

the conventional human-generated advisories and the frequently updated automated 

product. 

TMU respondents to the evaluation questionnaire indicated that the TCHP is most useful 

on the ARTCC scale, including a 150 nautical mile buffer outside of the ARTCC 

airspace. A conterminous US graphic would be useful with zoom and roam capability, 

applied to transcontinental overflight traffic decision-making. Most reported that ITWS 

is a superior display for gate and terminal area decision-making. 

The evaluation team recommends that work continue to improve the reliability of the 

storm motion and forecast position information provided by NCWF or subsequent 

versions of NCWF. 
21

Due to the number of comments regarding NCWF forecast reliability, the team 

recommends that the next phase of the evaluation include the enhanced version of the 

NCWF (described in the Summary), as well as exploration of the utility of including the 

CCFP. 

Since the NCWF Detection is a hybrid product containing derived radar information and 

lightning, and differs in appearance and information content from a conventional radar 

reflectivity display, some training for users is strongly indicated (see, for example, 

responses to question C4). 

6.3 General Recommendations 

Based on the findings of this assessment, the evaluation team recommends that all future 

automated forecast products include real-time performance information as part of the 

operational graphics package as a requirement for AWTT Level D5 approval. A 

performance graphic, such as is found on the ADDS javascript presentation of NCWF, or 

a similar real-time verification graphic, facilitates the intelligent application of, and 

confidence in, the product by end users. 

Based on the limited positive responses from users, refinement of the Impacted Jet Route 

product according to the feedback obtained, and additional evaluation of its utility in 

tactical decision making are justified.
22

APPENDIX A. RESPONSES FROM EVALUATION 

QUESTIONNAIRE 

Part A. TCHP Geographic Scales 

A1) CONUS Scale Graphic 

 a) Useful? Yes: 4 No: 9 No Opinion: 1 

 b) Easy to use? Yes: 7 No: 2 No Opinion: 5 

 c) Readable? Yes: 7 No: 2 No Opinion: 5 

 

Comments: 

(2003-07-05): Ability to Zoom In and roam to other areas of airspace. 

A2) ARTCC Scale Graphic 

 a) Useful? Yes: 14 No: 0 No Opinion: 0 

 b) Easy to use? Yes: 14 No: 0 No Opinion: 0 

 c) Readable? Yes: 14 No: 0 No Opinion: 0 

 Comments: 

(2003-06-26): Better if the product had sector high and low boundaries. 

(2003-06-27): Needs to have pan and zoom capability. 

(2003-06-26): Need Airport Diagrams. 

(2003-07-05): Growth and Decay Indicator/Forecast 

A3) Arrival Gate Scale Graphic 

 a) Useful? Yes: 6 No: 4 No Opinion: 4 

 b) Easy to use? Yes: 11 No: 1 No Opinion: 2 

 c) Readable? Yes: 11 No: 1 No Opinion: 2 

Comments: 

(2003-06-27): More detail is needed. Add the airport diagrams for DFW, AFW, 

FTW, DAL, NFW, GKY, RBD. Use standard symbology for VOR locations. 

Indicate outer marker and approaches to runways 36l/r 18l/r 31l/r 13l/r 36c. 

(2003-06-26): Better if the airport diagrams for 

DFW/DAL/AFW/FTW/NFW/GKY/RBD/ADS were available. 

(2003-06-27): ITWS has a better presentation, more flexibility with maps and user 

selectability. 

(2003-06-26): Need better map backgrounds such as Airport diagrams; also use 

the standard VOR symbols. ITWS is a better product at this point that the TCHP. 

(2003-07-02): ITWS provides more information and is updated more frequently. 

(2003-07-05): ITWS shows more information and is updated more frequently. 

(2003-07-05): Rwy Diagrams/Departure Routes/Fixes standardized 

(2003-07-06): ITWS seems more reliable. 

(2003-07-06): More details are required at this scale. More features similar to 

ITWS and include range rings. 
23

Part B. TCHP Graphical Components 

B1) Map Backgrounds 

 a) Are the map backgrounds adequate? 

 Yes: 14 No: 0 No Opinion: 0 

 b) Are the optional map background choices adequate? 

 Yes: 5 No: 9 No Opinion: 0 

 c) Are the map background colors satisfactory? 

 Yes: 0 No: 0 No Opinion: 0 

 Comments: 

(2003-06-14): Include sector boundaries and numbers on a toggle switch. 

(2003-06-27): Add capability to select and deselect jetroutes 

(2003-06-26): Need additional map backgrounds as stated above. In addition to 

what has already been noted the DFW departure corridors need to be added to the 

display. Also, the VORs need to use the standard FAA and NOAA symbology 

instead of an X. 

(2003-06-27): Need more adaptable features such as user selectable jetroutes. 

(2003-06-26): Need user selectable jetroutes available such as on the ETMS. 

Adding the aircraft targets would help greatly in making decisions. 

(2003-07-02): Additional map backgrounds are needed such as airport diagrams 

for DFW/AFW/FTW/NFW/DAL/ADS/GKY. Also you need to add user 

selectable jetroutes such as is available on the ETMS system. 

(2003-07-05): User Selectable Routes Including Jet Routes/Victor Airways/Q 

Routes 

(2003-07-06): Need to add more maps i.e. airport diagrams for the following 

airports DFW,DAL,AFW,FTW,NFW,ADS,RBD,GKY. Add user selectable 

jetroutes; make aircraft targets available with a one-minute refresh rate. 

(2003-07-06): Add airport diagrams, use the real VOR symbols. Give us a pan 

and zoom capability, and add aircraft like TSD. 

B2) Display Background Color 

 a) Is the display background color (neutral gray) satisfactory? 

 Yes: 14 No: 0 No Opinion: 0 

 Comments: 

(2003-07-05): Not a distraction. Very good neutral color. 

B3) NCWF Detection 

 a) Are the image colors satisfactory?

 Yes: 14 No: 0 No Opinion: 0 

 b) Is the number of levels of information adequate? 

 Yes: 13 No: 1 No Opinion: 0 

 

Comments: 
24

(2003-06-14): Intensity filters would be good. 

(2003-07-05): More levels similar to ITWS would be desirable. 

(2003-07-05): Colors conform with most weather products currently used. 

B4) NCWF 1-Hr Fcst 

 a) Graphic color satisfactory? 

 Yes: 14 No: 0 No Opinion: 0 

 b) Graphic readable? 

 Yes: 12 No: 2 No Opinion: 0 

 

 Comments: 

(2003-06-14): Accurate direction and speed data is more important. 

B5) Tops and Movement 

 a) Graphic color satisfactory?

 Yes: 14 No: 0 No Opinion: 0 

 b) Graphic readable?

 Yes: 14 No: 0 No Opinion: 0 

 c) Font size satisfactory? 

 Yes: 14 No: 0 No Opinion: 0 

 Comments: 

(0000-00-00)1

: Data is often obscured by plotted NCWD. 

(2003-06-26): It would be very beneficial to have the radar echo tops added to the 

display. 

B6) Convective SIGMET Nowcast 

 a) Graphic color satisfactory?

 Yes: 12 No: 0 No Opinion: 2 

 b) Graphic readable? 

 Yes: 12 No: 0 No Opinion: 2 

 c) Font size satisfactory? 

 Yes: 12 No: 0 No Opinion: 2 

(2003-06-14): This data is nice, but is rarely used. 

(2003-07-05): Haven't had enough applications to judge adequately. 

(2003-07-06): Displace text for SIGMET so it does not overwrite NCWD. 

B7) Convective SIGMET Forecast 

 a) Graphic color satisfactory?

 Yes: 13 No: 0 No Opinion: 1 

 b) Graphic readable? 

 Yes: 13 No: 0 No Opinion: 1 

 c) Font size satisfactory?

 

1

 All entries indicating a date of (0000-00-00) are from one interview, which occurred in late April, with the 

date inadvertently omitted on the evaluation form. 
25

 Yes: 13 No: 0 No Opinion: 1 

Comments: 

(2003-06-14): This data is nice, but is rarely used. We do not use sigmet boxes to 

plan for traffic or reroutes. 

(2003-07-05): Same as above. 

(2003-07-06): Same as B6 

B8) Convective SIGMET Text 

 a) Text font size satisfactory? 

 Yes: 7 No: 5 No Opinion: 2 

 b) Text overlay useful?

 Yes: 3 No: 9 No Opinion: 2 

 Comments: 

(0000-00-00): Not used. 

(2003-06-14): This data is nice, but is rarely used. We do not use sigmet boxes to 

plan for traffic or reroutes. 

(2003-06-26): Not useful. 

(2003-06-27): Never used! 

(2003-06-26): Never use the SIGMET text. 

(2003-07-02): Product is not used. 

(2003-07-05): Never use this product. 

(2003-07-05): Limited use, but size is acceptable. 

(2003-07-06): WITH WX PRESENT THE TEXT IS UNREADABLE 

(2003-07-06): Never use this. 

(2003-07-06): Never use. 

Part C: Utility of TCHP to TMU Operations 

C1) Is it useful to see this combination of graphical information? 

(2003-04-23) Yes 

(0000-00-00) Yes 

(2003-06-14): yes 

(2003-06-27): Yes. 

(2003-06-26): The combination is adequate but does not go far enough. 

(2003-06-27): Yes. 

(2003-06-26): Yes. 

(2003-07-02): Yes. We require more options. 

(2003-07-05): Yes. 

(2003-07-05): Yes, and the options available are good at this point however could 

be improved. 

(2003-07-06): YES 

(2003-07-06): YES 

(2003-07-06): Yes 

(2003-07-06): Yes. 
26

C2) What would improve the usefulness of the TCHP? 

(2003-04-23): Animation of NCWD 

(0000-00-00): Animation of NCWD 

(2003-06-14): Intensity filter, sector boundary overlays with a toggle for sector 

numbers 

(2003-06-27): The addition of aircraft targets. 

(2003-06-26): The product would be even more useful if the user could 

select/deselect individual jetroutes as on the ASD. Also, aircraft target 

information should be added to the product which would aid TMU in the decision 

making process. 

(2003-06-27): Sector maps need to be added to the map backgrounds. Add airport 

diagrams for DFW/AFW/FTW/DAL/NFW/GKY/RBD/ADS. Use the standard 

symbols for indicating VOR locations. 

(2003-06-26): Add the CCFP to the loop of the NCWD/NCWF, also make it a 

selectable product. 

(2003-07-02): Improvements to the system would be as follows: Add pan and 

zoom capability, include aircraft target data points as a selectable map or feature. 

Include CCFP as part of the product, this would help planning for routes used by 

TRANSCON aircraft. Using all of these features together would make this a 

good decision making tool. 

(2003-07-05): Add the following: 1. Aircraft target information. 2. Airport 

Diagrams for DFW/AFW/NFW/ADS/DAL/RBD/GKY/FTW. 3. Use the standard 

FAA symbols for VOR locations instead of the X. Add the CCFP forecast 

product to the NCWF, this would add the TMU traffic managers when setting up 

TRANSCON routes that can 4 hours or longer into the future. 

(2003-07-05): Ability to display TSD data. Growth and Decay indicators. Ability 

to add necessary NAS elements (i.e. jet routes, fixes, airport diagrams..). 

(2003-07-06): THE ABILITY TO ZOOM IN ON SPECIFIC AREAS AND THE 

ABILITY TO OVERLAY CENTER SECTOR BOUNDARIES 

(2003-07-06): THE ABILITY TO ZOOM IN ON A SPECIFIC AREA 

(2003-07-06): As stated above extra maps are a must. Include departure corridors 

on the TRACON display maps. Pan and zoom capability would be a definite 

improvement. Add the CCFP so TMU can look from 0 hour out to 6 hours. 

(2003-07-06): Add CCFP this would make the tool very useable for Strategic Plan 

of Operations Telcons. 

C3) Did the presence of NCWD-detected convective hazard in or near your 

airspace influence your decision making in this case? Please explain. 

(2003-04-23): Not really. More detailed information was gleaned from radar data 

provided by WARP and ITWS. 

(0000-00-00): Not really. Cells were isolated enough that the north bound 

departures were not impacted. 
27

(2003-06-14): yes, accurate cell movement and tops data in an easy to read format 

is always useful in projecting weather. 

(2003-06-27): No 

(2003-06-26): Not solely, used in combination with other radar and ASD 

information. 

(2003-06-27): Using the NCWD in a loop helps to verify other products such as 

WARP and ITWS. Did not make any operational decisions based on NCWD 

since it is not an operational product. 

(2003-06-26): No, used other sources of information that are operational to base 

decisions from. 

(2003-07-02): No, this is not an operational system. WARP, ITWS and ASD 

were used in the decision making process. 

(2003-07-05): No. This is not an operational product other sources such as 

WARP NEXRAD and ITWS are used. 

(2003-07-05): Decisions were made based on NCWD and forecast 

TOGETEHER! Very useful tool that validates CWSU forecasts. 

(2003-07-06): YES, THE CLEAR PRESENTATION SHOWED OPENINGS IN 

THE WX, AND THEREFORE THE ROUTE WAS NOT CLOSED...MILES-INTRAIL WERE USED FOR DEVIATIONS 

(2003-07-06): YES, IT GAVE A MUCH CLEARER DEPICTION OF THE 

SITUATION 

(2003-07-06): Yes, confirmed other data sources. 

(2003-07-06): No. 

C4) Did you find agreement/disagreement of the NCWD with your other radar 

echo depictions (for example, NEXRAD on ETMS) in this case? 

(2003-04-23): Disagreement between WARP and NCWD 

(0000-00-00): No comparison was made. 

(2003-06-14): disagreement on slow moving TS speed and direction 

(2003-06-27): Agreement, however radar had more coverage, as indicated in 

training. 

(2003-06-26): More data is available from the NEXRAD on WARP. 

(2003-06-27): Other radar displays show more detail and therefore are more 

trusted sources. 

(2003-06-26): NEXRAD on WARP BTs is more accurate and detailed. If there 

were a way of making a one hour forecast from the NEXRAD data that would be 

great. 

(2003-07-02): More detail was available from other sources such as WARP and 

ITWS. 

(2003-07-05): More detail is depicted by the NEXRAD on WARP. 

(2003-07-05): Yes however, NEXRAD is still more detailed. 

(2003-07-06): AGREEMENT...THIS PRODUCT IS MUCH CLEARER 

(2003-07-06): YES 

(2003-07-06): There is disagreement in the small scale. 

(2003-07-06): NEXRAD has more detail than NCWD. 
28

C5) Did the presence of NCWF-forecasted convective hazard in or near your 

airspace influence your decision making and/or result in any changes in your 

short-term planning in this case? If yes, please describe. 

(2003-04-23): NCWF was not reliable enough to make tactical decisions with, it 

either took too long for forecast to become available or the movement was 

inconsistent with other data sources such as ITWS and WARP. 

(0000-00-00): Same as C3. 

(2003-06-14): yes, see above. However, we never use any one product in decision 

making always try to use combination if available. Weather , as you know, is to 

unpredictable. 

(2003-06-27): Somewhat of an aid. 

(2003-06-26): NCWF is somewhat difficult to use in a tactical setting since not all 

areas have reliable movement depicted from NCWF. 

(2003-06-27): NCWF is not reliable enough to make tactical air operations 

decisions on. One set of graphics there and another set it is not. Sometime it 

takes too long to produce a forecast. 

(2003-06-26): NCWF is not reliable enough to make tactical air traffic decisions 

from yet. 

(2003-07-02): No, again this is not an operational system. Also, we noted that 

during this event the NCWF was 90 degrees off the actual movement of the cells. 

At approximately 2230z NCWF indicated movement from around 280 degrees at 

19 knots while the detected movement through looping and ITWS both indicated 

cells moving from around 020 degrees at 5 to 10 knots. 

(2003-07-05): Noticed a 180 degree difference in movement this morning at 

1131z in ZHU ARTCC airspace. This does not create confidence in the product. 

(2003-07-05): No, however ZFW is fortunate enough to have ITWS at 

DFW/DAL, and most of the decision at ZFW are based on these airports. If I 

didn't have ITWS this would be an acceptable tool for decision making. 

(2003-07-06): YES SEE C3 

(2003-07-06): IT REAFFIRMED MY THOUGHTS 

(2003-07-06): Somewhat, however NCWF is not as reliable as looking at a loop 

when cells are moving slowly or not at all, use together with SIGMET forecast 

seems to work better. 

(2003-07-06): No, but it did provide better information as to where the strongest 

thunderstorms were located. 

C6) Did the presence of a Convective SIGMET nowcast in or near your 

airspace influence your decision making in this case? If yes, please describe. 

(2003-04-23): No 

(0000-00-00): No 

(2003-06-14): No and never will. 

(2003-06-27): No 

(2003-06-26): No 
29

(2003-06-27): No. 

(2003-06-26): No 

(2003-07-02): Did not monitor this product. TMU does not make decisions based 

on SIGMET information. 

(2003-07-05): No. 

(2003-07-05): Not an influence at all. Not sure if any application of this product 

could be beneficial. Real time/near time is much more needed. 

(2003-07-06): NO 

(2003-07-06): YES 

(2003-07-06): Same as C5 

(2003-07-06): No. 

C7) Did the presence of a Convective SIGMET forecast in or near your 

airspace influence your decision making in this case? If yes, please describe. 

(2003-04-23): No 

(0000-00-00): No 

(2003-06-14): No and never will. 

(2003-06-27): Somewhat, it gave more believability to the NCWF. 

(2003-06-26): No 

(2003-06-27): No. 

(2003-06-26): No 

(2003-07-02): No, not an operational product. 

(2003-07-05): No. This product is never used to move aircraft in the tactical 

environment. 

(2003-07-05): NO. See above. 

(2003-07-06): YES BECAUSE IT RECONFIRMS MOVEMENT AND 

STABILITY OF THE WX 

(2003-07-06): YES, IT GAVE ME MORE INFORMATION TO MAKE A 

DECISION 

(2003-07-06): No. 

(2003-07-06): It aided and filled in holes left by the NCWF. 

C8) Did you find disagreement between the Convective SIGMET nowcast 

and/or Convective SIGMET forecast with the NCWF in this case? If yes, please 

describe. 

(2003-04-23): The convective sigmet forecast was more in line with actual 

movement of thunderstorms. NCWF was not always reliable therefore unusable in 

air traffic decision making. 

(0000-00-00): no 

(2003-06-14): no 

(2003-06-27): Some disagreement in the NCWF and SIGMET areas of NCWF 

showed movement 180 different than SIGMET. 

(2003-06-26): Convective Sigmet was better in catching early movement. Too 

many areas do not display movement making it hard to base decision on NCWF. 
30

The product is not ready to be used in an operational tactical setting. It might be 

acceptable at a strategic scale. 

(2003-06-27): Movement between the SIGMET and NCWF sometimes differ. 

SIGMET seems to be more in line with loops of NCWD. 

(2003-06-26): SIGMET NOWCAST seemed more reliable than the NCWF was at 

times. 

(2003-07-02): Yes. As stated above in C5. 

(2003-07-05): As in C5 movement 180 degrees different between the NCWF and 

SIGMET. 

(2003-07-05): N/A 

(2003-07-06): NO 

(2003-07-06): NO 

(2003-07-06): See C5 

(2003-07-06): Yes, NCWF in many cases is not available so we can use the 

SIGMET forecast as means of tracking movement. 

C9) Do you think that the TCHP contributed to safer, more efficient operations 

in this case? 

(2003-04-23): No. 

(0000-00-00): No opinion 

(2003-06-14): Yes and No, we do not use any one single product 

(2003-06-27): somewhat 

(2003-06-26): Too early to use with any confidence. 

(2003-06-27): Not at this point. 

(2003-06-26): Not used yet as an operational tool. 

(2003-07-02): No 

(2003-07-05): No 

(2003-07-05): Yes. 

(2003-07-06): YES 

(2003-07-06): YES 

(2003-07-06): Not enough thunderstorms in this case to make that determination. 

However having it TCHP where ITWS was not available aided in making 

decisions. 

(2003-07-06): To early to tell. 

C10) Is the TCHP 0-1 hour valid time adequate for your decision making? If 

not, what time span would you prefer? At what time intervals? 

(2003-04023): 0-2 hours. 

(0000-00-00): It would be helpful for the TCHP product to transition from 0 to 1 

hour and then display the CCFP. 

(2003-06-14): yes 

(2003-06-27): It is okay for the tactical, however the addition of the CCFP would 

make it a much better planning tool. 
31

(2003-06-26): NCWD is good information but NCWF is less reliable. It would 

also be a great asset to have the CCFP forecast added to make better decisions for 

TRANSCON flights. 

(2003-06-27): The loop helps, but the NCWF is not adequate enough to base 

aircraft movement decisions from. The product would be much better as a 

planning tool if the CCFP forecasts were added. 

(2003-06-26): 00-06 hours would be more beneficial this would help us to plan 

for TRANSCON flights based on forecast areas of weather. 

(2003-07-02): A one forecast would be good but based on the performance during 

this event I would not use the product. 

(2003-07-05): As mentioned above expanding this product to include the CCFP 

would be very valuable. The 0-1 hour product is good when used in a loop. 

(2003-07-05): Yes, however utilizing CCFP and growth and decay and giving the 

ability to select at least 3 hour intervals would be useful. 

(2003-07-06): 1-2 HR WOULD BE VERY BENEFICIAL 

(2003-07-06): 0-2 HOUR 

(2003-07-06): Add the CCFP so that we can see from present out 6 hours. This 

would be a great tool for planning TRANSCON routes, especially when used with 

aircraft target data. 

(2003-07-06): Add CCFP so we can look from the present out to 6 hours. 

C11) Did you have any evidence that the convective forecast was 

accurate/inaccurate? What was the evidence? 

(2003-06-14): radar loop 

(2003-06-27): ADD CCFP. 

(2003-06-26): see above 

(2003-06-27): see above 

(2003-06-26): Add the CCFP and aircraft target information to the display. 

(2003-07-02): Improve the forecast movement of NCWF. Also, add to the TCHP 

the CCFP forecast along with aircraft target information. 

(2003-07-05): See C10. Also, add the aircraft target information from ASD. 

(2003-07-05): See comments above. 

(2003-07-06): THOUGHT IT WAS SATISFACTORY 

(2003-07-06): CCFP. 

(2003-07-06): More detail in the NCWD, improve the NCWF movement. 

C12) User Documentation Adequate? 

a) NCWD/NCWF

Yes: 11 No. 0 No Opinion: 3 

b) Convective SIGMET

Yes: 11 No. 0 No Opinion: 3 

c) Web site/User Interface 

Yes: 11 No. 0 No Opinion: 3 
32

Comments: 

(2003-07-05): Information was presented thoroughly, however Convective 

SIGMET application is not useful. 

C13) Training adequate? 

Yes: 11 No: 0 No Opinion: 3 

Comments: 

(2003-06-27): Training was very good. 

(2003-06-26): Dennis did a great job. 

(2003-07-02): Training was thorough. 

(2003-07-06): Very Good. 

Part D. Impacted High Jet Route Product 

D1) Is this display useful in support of tactical decision making? 

Yes: 4 No: 2 No Opinion: 6 

Comments: 

(2003-06-26): Not quite ready to used operationally. 

(2003-06-27): Not ready for operational use in the tactical environment yet there 

would be some benefit in the strategic if you add the CCFP to the TCHP. 

(2003-06-26): NCWF is not accurate or reliable enough to use operationally yet. 

(2003-07-02): Not ready for operational tactical movement of aircraft. 

(2003-07-05): Haven't used tool yet. 

(2003-07-06): With more refinements this will be a good tool. 

D2) This product displays high use jet routes. Would the display be more useful 

or less useful with all jet routes displayed? 

More Useful: 3 Less Useful: 9 

Comments: 

(2003-06-27): Add user selectable jetroutes, so that TMU can select or deselect 

for given situations and weather/traffic problems of the day. 

(2003-06-27): Make jetroutes that are user selectable. 

(2003-07-02): Need to add a button so the user can select the desired jetroutes to 

be displayed. 

(2003-07-05): Only if routes are user selectable. 

(2003-07-06): Give user selectable jetroutes. 

D3) "Impacted" for this product is defined as touching the jet route. What 

width buffer zone should be included in "impacted"? 

a) 0 NM (touching, as it is now) 1 

b) 5 NM either side of the line 0 

c) 10 NM either side of the line 11 

d) Other; please specify below.
33

Comments: 

(2003-06-14): Impacted is a subjective term, 15 miles 

D4) "Impacted" for this product is defined as above 17,000ft MSL. What 

vertical extent should be included in "impacted"? 

a) Above 17,000 ft MSL (as it is now) 3 

b) Above 17,00 ft MSL to the NCWF Tops 9 

c) Above 17,00 ft MSL to some buffer 

above the NCWF Tops; please specify 

buffer below. 0 

(no comments entered) 

D5) "Impacted" for this product is defined as touched by Level 3 or Greater 

NCWF Hazard Detection (Red), or touched by any portion of a NCWF 1-hr 

Forecast (Yellow). 

a.) Would you include C-SIGMET Nowcasts as an "Impact"? 

Yes: 1 No: 11 No Opinion: 0 

b.) Would you include C-SIGMET 1-hr Forecast as an "Impact"? 

Yes: 3 No: 9 No Opinion: 0 

Comments: 

(2003-06-27): Decisions for air traffic have never been made from the SIGMET. 

D6) Is the graphical presentation of the Impacted Jet Routes Product 

satisfactory? 

Yes: 4 No (please specify); 6 No Opinion: 2 

Comments: 

(2003-06-27): Have the right idea see comments above for additions to make the 

product better. 

(2003-06-26): Thicken the lines 

(2003-07-02): Lines need to be thicker. 

(2003-07-05): Make the lines thicker. 

(2003-07-05): Line thickness should be increased. See additional comments 

above. 

(2003-07-06): Make the jet route lines thicker.