**Graphical Hazard Generation/Valid Time Event Code Operational Test and
Evaluation**

**Assessment Report**

**August 19, 2005**

**Prepared by the**

**National Weather Service**

**Office of Science and Technology**

**Office of Climate Water and Weather Services**

**Office of Operational Systems**

**Graphical Hazard Generation/Valid Time Event Code Operational Test and
Evaluation**

**Assessment Report**

**August 19, 2005**

**Approved:**

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ \_\_\_\_\_\_\_\_**

Gregory A. Mandt, Director Date

Office of Science and Technology

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ \_\_\_\_\_\_\_\_**

Dennis H. McCarthy, Director Date

Office of Climate, Water, and Weather Services

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ \_\_\_\_\_\_\_\_**

Louis W. Uccellini, Director Date

National Centers for Environmental Prediction

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ \_\_\_\_\_\_\_\_**

John McNulty, Director Date

Office of Operational Systems

# Table of Contents

[Executive Summary 1](#executive-summary)

[1. Introduction 3](#introduction)

[1.1. Background 3](#background)

[1.1.1. VTEC 3](#vtec)

[1.1.2. Graphical Hazard Generation (GHG)
3](#graphical-hazard-generation-ghg)

[1.2. Purpose 4](#purpose)

[1.3. Test Objectives 4](#test-objectives)

[2. Test Results 5](#test-results)

[2.1. Test Activities 6](#test-activities)

[2.2. Product Evaluation 7](#product-evaluation)

[2.3. Product Evaluation Criteria 8](#product-evaluation-criteria)

[2.4. Product Evaluation Results 8](#__RefHeading___Toc112061913)

[2.5. Failure Analysis 10](#failure-analysis)

[2.5.1. Marine Weather Statement (MWS)
10](#marine-weather-statement-mws)

[2.5.2. Flood/Flash Flood Watch Product (FFA)
11](#floodflash-flood-watch-product-ffa)

[2.5.3. Manual Editing of Products 11](#manual-editing-of-products)

[2.5.4. Operator Errors (training issues)
12](#operator-errors-training-issues)

[2.5.5. Local Configuration Issues 12](#local-configuration-issues)

[2.5.6. GHG Software Errors 12](#ghg-software-errors)

[2.5.7. AWIPS Software 13](#awips-software)

[2.6. Training Provided 13](#training-provided)

[2.6.1. Training evaluation criteria 14](#training-evaluation-criteria)

[2.6.2. Training Survey Results 15](#training-survey-results)

[2.7. GHG Concept of Operations 15](#ghg-concept-of-operations)

[3. Issues and Recommendations 16](#issues-and-recommendations)

[3.1. Watch-By-County Issues 16](#watch-by-county-issues)

[3.1.1. Product Errors 16](#product-errors)

[3.1.2. Extending Watches 16](#extending-watches)

[3.1.3. WOU Update Product 16](#wou-update-product)

[3.2. Marine Weather Statement Issue
17](#marine-weather-statement-issue)

[3.3. Software Stability Issues 17](#software-stability-issues)

[3.4. Service Backup Issues 18](#service-backup-issues)

[3.5. Headline Formulation Issues 18](#headline-formulation-issues)

[3.5.1. Requirements 18](#requirements)

[3.5.2. Issues 19](#issues)

[3.6. Additional Problems Identified by OT&E Testing
19](#additional-problems-identified-by-ote-testing)

[4. Summary and Conclusions 20](#summary-and-conclusions)

[Appendix A: OCONUS Testing 21](#appendix-a-oconus-testing)

[Appendix B: Sample Test Case Scenario
24](#appendix-b-sample-test-case-scenario)

[Appendix C: Test Review Group 27](#appendix-c-test-review-group)

[Appendix D: Offices Participating
29](#appendix-d-offices-participating)

[Appendix E: Acronyms 30](#appendix-e-acronyms)

# Executive Summary

The NOAA National Weather Service (NWS) conducted an Operational Test
and Evaluation (OT&E) of the Interactive Forecast Preparation System
(IFPS) Graphical Hazard Generation (GHG) capability to produce watch,
warning and advisory products with Valid Time Event Coding (VTEC). This
OT&E focused on a subset of products that require VTEC coding and will
be produced using the GHG software. Additional testing is planned during
the fall of 2005 to test the VTEC code implementation in the remaining
NWS products which require the use of other AWIPS applications (WARNGEN
and RiverPro).

The GHG/VTEC OT&E was conducted from 0000 UTC 9 May to 1200 UTC 24 June
2005. A total of 35 NWS Weather Forecast Offices (WFO) created
operational and test products using GHG in both the CONUS and OCONUS.
Additional non-operational testing was conducted at Pacific Region
Headquarters (PRH) and Alaska Region Headquarters (ARH).

A total of 5,223 products were transmitted to the public with VTEC
coding from 9 product categories. These products were evaluated for
accuracy in product format, headline format, proper MND heading coding
and proper VTEC action, phenomena and event tracking number coding. A
product was only considered successful if it passed all of these tests
in all segments of the product. An overall success rate of 97.3%
occurred with the use of GHG software.

Based on the outcome of the OT&E, the Test Review Group (TRG) makes the
following recommendations to the NWS Corporate Board:

1.  Send a Service Change Notification message on June 30, 2005 of the
    NWS intent to make VTEC operational in the following products on
    November 1, 2005:

Watch County Notification Message (WCN)

Watch Outline Update (WOU update)

Flood/Flash Flood Watch (FFA)

Fire Weather Watch/Warning Message (RFW)

Winter Weather message (WSW)

Non-Precipitation Weather Message (NPW)

Coastal Waters Forecast (CWF)

Coastal/Lakeshore Hazard Message (CFW)

Near Shore Forecast (NSH)

2.  Delay the operational implementation of VTEC in the Marine Weather
    Statement (MWS) product for the MA.S VTEC phenomena code until after
    successful testing using the WARNGEN AWIPS application. Note: The
    severe convective follow-up MWS product for the MA.W VTEC phenomena
    code has been operational since February 8, 2005.

This phenomenon code has both a short-term (1 hour or less) and
longer-term needs. Although GHG properly produced VTEC for the
longer-term statements, short-term, non-severe convective events require
the use of the WARNGEN software so the forecaster can utilize nowcasting
tools and produce short duration statements. Current WARNGEN software
does not provide VTEC for this type of statement. This software will
need to be upgraded and tested for this purpose.

3.  Concerns about software stability and the use of the "quick" service
    backup method came up during the OT&E. Solutions to software
    stability problems have been identified and tested. The TRG
    recommends that these solutions be implemented as soon as possible.
    The TRG also recommends that additional testing of the "quick"
    service backup method be conducted using updated IFPS software to
    ensure the process works properly.

# 1. Introduction

## Background

### VTEC

Planning for implementation of a Valid Time Event Code (VTEC) began in
1997 under the Office of Meteorology to improve usability of NWS Watch,
Warning and Advisory products. VTEC is a code that describes the content
of watches, warnings and advisories, (i.e. what event is being alerted
and when the event is effective.) The standardized coding allows
selective dissemination, reception and display of NWS text products. In
addition, NWS customers and partners can use VTEC to follow the life
cycle of every watch, warning, or advisory from when it is issued until
it either expires or is canceled.

The implication of such coding is that it removes human intervention and
judgment from the receipt and dissemination process thereby allowing
messages to be promulgated to the public for personal and business use
and in some cases, initiating significant actions to protect life and
property. It is this implication that requires these messages to be
accurate in terms of internal consistency between the VTEC string and
the product text as well as delivery of an intended message.

The objectives of the VTEC Program are to:

1)  Implement VTEC in NWS Watch, Warning and Advisory products,

2)  Improve interpretation and accessibility of critical NWS products,

3)  Reduce errors in critical NWS products, and

4)  Improve end-to-end tracking of products associated with specific
    meteorological and hydrological events.

The VTEC specification, NWS Instruction (NWSI) 10-1703, defines the code
and proper message formatting. On February 8, 2005, VTEC coding was made
operational for the initial convective watch product (WOU) and for the
following short-fused warning products; Severe Thunderstorm Warning
(SVR), Tornado Warning (TOR), Severe Weather Statement (SVS), Special
Marine Warning (SMW), and Marine Weather Statement (MWS) when issued as
a follow-up to the SMW.

### Graphical Hazard Generation (GHG)

The NWS has transitioned the production of forecast products to the
digital age through the creation of a high resolution (5 km or less)
digital forecast database. A key element of the NWS Interactive Forecast
Preparation System (IFPS) is the production of forecast grids for each
weather element (temperature, dewpoint, etc.) using the Graphical
Forecast Editor (GFE) program. In the fall of 2003, the NWS completed an
initial operating capability in the CONUS for the production of NWS text
products using product formatters based on this digital forecast
database. Some products (e.g., zone forecast) require headlines that
highlight long-fused watches and warnings that are in effect. Using GFE,
forecasters began creating a hazards grid for this purpose.

A new capability for the issuance of WWA products which utilized the
digital forecast grids surfaced in the Central Region and was prototyped
during the winter of 2003-2004 at 20 Central Region offices. It was
initially called the Graphical Headline Generator (GHG). Matt Davis, ITO
at WFO Lacrosse WI developed this prototype and shared it with the
Forecast Systems Laboratory (FSL). It is this concept that was
integrated into the IFPS 17 GFEsuite software that was tested and used
in the Operational Test and Evaluation (OT&E). GHG (now short for
Graphical Hazards Generation) refers to the capability in the IFPS
software to produce and disseminate watch, warning and advisory
products.

## Purpose

The OT&E was held from May 9, 2005 to June 24, 2005. The OT&E was
conducted under the guidelines of the GHG/VTEC OT&E Plan dated May 5,
2005. Number range CHAPTERThe purpose of the OT&E was to determine the
operational effectiveness and suitability of IFPS GHG software under
realistic conditions, and to determine if specified minimum acceptable
operational performance requirements have been satisfied. The goal of
the OT&E was to demonstrate and validate the NWS' readiness to release
operational messages with the VTEC string implemented. This report
summarizes the OT&E test activities, test results, issues and
recommendations of the Test Review Group (TRG).

## Test Objectives

An assessment of the operational capabilities of VTEC was conducted for
only those products that are issued using the AWIPS IFPS GHG software.
The assessment was accomplished through validation of the following test
objectives using the evaluation criteria listed in section 3.

-   Demonstrate that accurate VTEC codes can be generated in operational
    products in an operational environment under real world weather
    situations and scripted scenarios.

```{=html}
<!-- -->
```
-   VTEC codes conform to the specification defined in NWSI 10-1703

    -   VTEC strings can be accurately generated when WFOs are in
        service backup mode.

```{=html}
<!-- -->
```
-   Demonstrate training was provided to meet the objectives defined for
    the VTEC program.

    -   Training addressed the key components necessary to successfully
        issue watches, warnings, advisories and follow-up statements
        with VTEC strings.

```{=html}
<!-- -->
```
-   Demonstrate the VTEC concept of operations delivered in GHG is
    viable.

    -   Forecasters use GHG to produce hazards grids such that automated
        VTEC features yield correct interpretation by customers.

    ```{=html}
    <!-- -->
    ```
    -   The GHG software is usable for the reliable generation and
        issuance of watch, warning, and advisory products and system
        performance does not appear to be degraded during hazardous
        weather or service backup operations.

    ```{=html}
    <!-- -->
    ```
    -   Implementation of VTEC appears to cause no significant increase
        in forecaster workload when compared to other product generation
        software (e.g., XNOW, WWA).

```{=html}
<!-- -->
```
-   Demonstrate that VTEC quality control functionality is implemented
    effectively.

    -   Products are standardized in accordance with the requirements of
        the VTEC specification.

```{=html}
<!-- -->
```
-   Demonstrate VTEC can be read and processed by NWS customers and
    partners.

    -   The intended watch or warning could be ingested and decoded.

```{=html}
<!-- -->
```
-   Demonstrate the WBC concept of operations delivered in GHG software
    is viable.

    -   WFOs can issue, update, cancel, extend in time, extend in area
        and extend in both time and area watches using the WCNs for
        their geographic area of responsibility. WCNs contain proper
        Universal Geographic Codes (UGC), Valid Time Event Code (VTEC)
        and list of counties, independent cities and marine zones (as
        appropriate) for each segment of the product.

    ```{=html}
    <!-- -->
    ```
    -   WFOs can combine more than one active convective watch into a
        segmented WCN product.

    ```{=html}
    <!-- -->
    ```
    -   Affected WFOs can "clear" all counties and independent cities
        from a convective watch for their geographic area of
        responsibility using a WCN product.

# Test Results

The OT&E was conducted at selected field sites (figure 1) to verify VTEC
implementation in real-time service operations. VTEC coded products were
generated using IFPS GHG product generation software when actual
meteorological and/or hydrologic conditions warranted the issuance of
any watch, warning or advisory. Also, the OT&E sites were instructed to
perform test case scenarios to generate test VTEC coded products on a
non-interference basis. These artificially generated products were coded
with the "/T" VTEC prefix and included test wording in the product
heading to indicate it as being a test product. This activity was
designed to verify VTEC in products for weather phenomena that are
seasonal (or occur much less frequently during the OT&E period) and may
not occur in real-time.

GHG was tested in both the routine and service backup modes of
operation. Full service backup is routinely coordinated and invoked at
the backup office for scheduled AWIPS software and hardware updates.
These updates usually require the system to be down for several hours or
more at the primary office. Updates to AWIPS software and hardware
occurred at 16 OT&E sites during the OT&E period. Seven of these offices
required two different service backup periods, one for installation of
software for the use of the new DX hardware and a second time to upgrade
the baseline AWIPS software to version OB4.2. Service backup was also
tested in several test case scenarios (see example scenario in Appendix
B).

![](media/image1.jpeg){width="6.495833333333334in"
height="4.410416666666666in"}

Figure 1: Location of offices participating in the GHG/VTEC OT&E
(colored in circles)

## Test Activities

The regional and service program focal points collected and evaluated
the VTEC products produced by the test sites. The products were captured
electronically at the NWS Telecommunication Gateway and made available
to the Test Review Group (TRG) (see Appendix C) through a web interface.
The regional and service program focal points interacted with the point
of contact (POC) at the OT&E site to learn more about the causes of
identified problems. Frequent conference calls were held to coordinate
test activities and feedback between the OT&E sites and the TRG.
Identified problems were logged into the Test Trouble Report (TTR)
database and reviewed by the TRG. Products evaluated during this OT&E
were:

-   WCN -- Watch County Notification

-   NPW -- Non-Precipitation Weather Message

-   WSW -- Winter Weather Message

-   FFA -- Flood/Flash Flood Watch

-   CFW -- Coastal/Lakeshore Hazard Message

-   CWF -- Coastal Waters Forecast

-   MWS -- Marine Weather Statement

-   NSH -- Near Shore Forecast

-   RFW -- Fire Weather Watch/Warning Message

![](media/image2.png){width="4.125in" height="3.972916666666667in"}

Figure 2: Percentage of all products evaluated by product type

A total of 5,223 products were issued and evaluated during the OT&E. The
percentage of the total issued and evaluated by product category is
shown in figure 2. The CWF product had the highest percentage of the
total (43%). This occurred because the CWF is a routine marine coastal
waters forecast product that was issued at least 4 times a day by 10 of
the 35 OT&E offices.

## Product Evaluation

Each product was checked by at least one TRG member and was scored as
either passing or failing the evaluation. The product was listed as
failing if any segment of the product did not meet all of the following
evaluation checks:

a)  Did the VTEC event beginning time and/or event ending time match the
    times and (or) time phrases in the headline(s)?

b)  Did the product format conform to the latest NWS Instructions?

c)  For first issuance of a new event:

    a.  Was the VTEC Event Tracking Number (ETN) properly incremented
        from the previous issuance of this phenomenon?

    b.  Was the VTEC action code NEW?

    c.  Did the VTEC event beginning date/time group have valid
        date/time information?

d)  For an on-going event:

    a.  Was an appropriate ACTION code used for each UGC code that was
        contained in the previous product (CON, CAN, EXP, EXT)?

    b.  Was an appropriate ACTION code used for the inclusion of
        additional UGC (EXA, EXB, NEW)?

    c.  Did the ETN remain the same for this phenomena code?

    d.  Did the expiration time remain the same? (CON, EXP, CAN, EXA)

e)  For an upgrade of an event, was there a pair of VTEC lines with
    proper Action codes (UPG & NEW, UPG & EXA, UPG & EXT or UPG & EXB)?

f)  For a downgrade of an event, were there a pair of VTEC lines with
    proper Action codes (CAN & NEW, CAN & EXA, CAN & EXT and CAN & EXB)?

g)  For corrections to a product, did the VTEC correction Action code
    (COR) match the corrected BBB field (CCx) in the WMO heading?

## Product Evaluation Criteria

VTEC code compliance was evaluated by the inspection of the VTEC strings
in products issued by OT&E offices during the test period. Every field
of the VTEC strings was checked. One error, or multiple errors, in the
product's VTEC string(s) or a lack of internal consistency within the
product was cause for failing the product

[Success criteria]{.underline}:

1.  The format of VTEC String matches NWSI 10-1701 and NWSI 10-1703.

2.  Product type (O,T, E, X) and Mass News Disseminator (MND) header are
    consistent.

3.  Event Tracking Numbers are continuous in normal operations and in
    service backup mode.

4.  Corrections are accurately reflected in the BBB field and VTEC, UGC
    and Headlines are consistent within the product.

[Success]{.underline}: Products pass these checks 95% of the time
(tolerance, 5%).

[Failure]{.underline}: aggregate of products with error is more than 10%
of total products issued during the period.

1.  []{#__RefHeading___Toc112061913 .anchor}***Product Evaluation
    Results***

For all products combined, the product success rate was 97.3% for the
entire OT&E. Figure 3 shows the total number of products issued and the
number of failed products stratified by each week of the OT&E. Products
issued include operational weather dependant products and test case
scenario test products.

![](media/image3.wmf){width="4.714583333333334in"
height="3.3020833333333335in"}

Figure 3: Number of products issued (blue) and number of products with
errors (red) stratified by week during the OT&E.

Table 1 shows the overall success rates for each product. The data are
also stratified by CONUS test sites and OCONUS test sites. The intent of
testing was to implement VTEC coding at CONUS sites. However, Fairbanks
Alaska was also included as a test site to determine the viability of
using this software at OCONUS NWS offices. (See Appendix A for more
information concerning OCONUS testing).

![](media/image4.wmf){width="6.490972222222222in"
height="2.9243055555555557in"}

Table 1: Success rate by product type and by CONUS or OCONUS test sites

The test evaluation information was also stratified by week (Table 2) to
see any trends in the errors during the test period.

![](media/image5.png){width="6.491666666666666in"
height="3.222916666666667in"}

Table 2: Success rate stratified by week and product during the 7 week
OT&E period.

## Failure Analysis

A review of failed products showed several causes for the failures. When
a problem was noted with a product, the office was contacted to discover
the circumstances involving the issuance of the product. Some of these
problems were due to software issues while others were training related
or operator error issues. Nearly all failures can be grouped into one of
several categories discussed in this section. Each section indicates
actions that will be taken to avoid failures in the future.

###  Marine Weather Statement (MWS)

Only five MWS products were issued during the first week of the OT&E.
Three of these products were failed when evaluated. All three products
were issued with the VTEC "O" coding which signifies operational status
of the VTEC coding instead of the use of the "X" code to indicate the
experimental status of the VTEC code. All three products were sent from
the same office and were issued using a modified template for the AWIPS
WARNGEN application. These products were not generated using the GHG
software.

***Mitigation Action:*** The office was instructed to use the baseline
template for the MWS product generation in WARNGEN. Once this was
corrected, no other VTEC coding errors were noted during the OT&E in the
MWS product. See Section 3.2 for additional discussion concerning the
MWS product.

###  Flood/Flash Flood Watch Product (FFA) 

The poor success rate for the FFA product during the last 4 days of the
OT&E (64.7%) occurred when 5 products did not pass evaluation. An
additional test case scenario was designed to confirm suspicions about a
specific service backup situation. For the FFA product, an office can
use either the zone-based UGC coding or the county-based FIPS codes to
indicate the area within the watch. The Portland Oregon office (WFO PQR)
uses the zone-based codes and is the service backup office for the
Seattle Washington office (WFO SEW). WFO SEW uses the county-based codes
in the FFA product. Using the "quick backup" method of service backup in
GHG, the local configuration is utilized by the backup office when
issuing products for the other office. The result was the use of the
wrong UGC coding and also errors with the VTEC event tracking. Five
products were failed during this test and represents 25% of all FFA
errors during the OT&E.

GHG has two flavors of service backup. The system has a "full backup"
mode and a "quick backup" mode. The full backup mode downloads the
office configuration and the latest forecast grids from a central server
to the backup office. This transfer of data can take 10 to 20 minutes
depending on network loading conditions. The "quick backup" mode allows
the backup office to quickly configure GHG to produce products for the
failed office without having to download the configuration files or
forecast grids. In this test scenario, it was clear that the use of
local configuration in place of the failed site's configuration could
lead to errors.

***Mitigation Action:*** A redesign of the "quick backup" capability has
been done. The new capability will only download from the backup server
the latest site configuration data. This will allow the site to begin
issuing products in a timely manner -- much sooner than the "full
backup" option.

###  Manual Editing of Products

The success rate during the first week of the OT&E (93.0%) was the
lowest of any week of the OT&E and reflected spin-up issues for
forecasters at many of the OT&E offices. Most of the failed products
during the first two weeks were due to the manual editing (not using
GHG) of the product by the forecaster. This resulted in VTEC, headline
and product format errors.

Two reasons for manual editing emerged. First, forecasters were
unfamiliar with changes in the product formatting directives. The GHG
software was designed to reflect new requirements for product and
headline formats that were included in directives that were updated for
VTEC. Forecasters that were unfamiliar with these changes were
"correcting" their products to conform to the previous standard. In
addition, we found that some offices had an operating procedure for
creating products in "work files" and then re-editing them using the
AWIPS product editor (outside of GHG). These old editing habits could no
longer be allowed due to the time sensitivity and complexity of the VTEC
code.

***Mitigation Action:*** Updated training videos and the "GHG lessons
learned" web site
([[http://www.nws.noaa.gov/os/vtec/GHG_LL.html]{.underline}](http://www.nws.noaa.gov/os/vtec/GHG_LL.html))
will reflect these issues.

### Operator Errors (training issues)

A few common training issues emerged during the OT&E. Forecasters
occasionally were deleting hazards from old time periods of the hazard
grid in GHG. These older time periods are necessary for GHG to
understand about on-going events. When the forecaster deleted these
hazards, the software produced erroneous VTEC coding, such as a
cancellation (CAN) instead of an expiration (EXP) or an extension in
time (EXT) instead of a continuation (CON) in the VTEC code.

Another operator error was seen when the forecasters did not indicate
(using the GHG product editor interface) that the product was a
correction or retransmission. This resulted in errors in the Mass News
Disseminator (MND) heading of the product.

***Mitigation Action:*** Updated training videos and the "GHG lessons
learned" web site
([[http://www.nws.noaa.gov/os/vtec/GHG_LL.html]{.underline}](http://www.nws.noaa.gov/os/vtec/GHG_LL.html))
will reflect these issues.

### Local Configuration Issues

Some errors were found to be attributed to local configuration issues.
In most cases, product formatter overrides were found to be the cause of
the problem. These overrides were carried over from previous versions of
the software and needed to be removed to allow the software to properly
format the product. Also, updates were needed to the "area dictionary"
file to properly reflect local geographic information needed by the
product formatters.

***Mitigation Action:*** These configuration issues have been added to
the "VTEC Enabling Checklist". This checklist has been distributed to
the NWS offices that were not a part of the OT&E for their use in
preparing their office for VTEC and GHG. Completion of the checklist is
required by each office prior to the use of VTEC coding in operational
products.

### GHG Software Errors

Some product failures were traced to issues with the software. The main
problems identified were:

-   WCN product text bug: Product was missing the description of the old
    watch in certain watch replacement situations

-   Headline ellipse problem, occasionally only a "..." appeared without
    the headline

-   Fuzzy wording of headlines issues occurred with some hazard
    configurations (e.g., "from this morning until this morning")

-   Adjacent events in hazard grid which did not overlap in time
    resulted in upgrade (UPG) coding instead of NEW. This was seen with
    ramping down of marine hazards.

-   UGC marine zone codes were mixed with FIPS county codes in same
    segment of WCN products. This violated NWSI 10-1701

-   Confusion by the forecasters in how to set the expiration times of
    the products using the interface.

***Mitigation Action:*** These software issues have been corrected in
updated versions of IFPS software (IFPS 17.5 and 17.6) which will be
deployed before the operational implementation of VTEC.

### AWIPS Software 

Some product failures were traced back to issues with the AWIPS system.
The failover process between the DX servers was not executing properly
and resulted in essential processes not being restarted. The critical
GHG related process is called "NotifyTextProd". This process triggers
the VTEC decoder when products are received. Without VTEC decoding, the
software cannot properly track ongoing events.

Another issue was the mix of hardware and software configurations in
place at the test sites. An AWIPS hardware refresh was in progress
during the OT&E -- upgrading the application servers (PX) with newer
technology (DX). This resulted in several versions of AWIPS software
(4.1, 4.2, and 5.0) which utilized different configurations of hardware
(4.1-PX, 4.1-DX, 4.2-DX and 5.0-DX). The DX system servers have a newer
version of the Linux operating system (RedHat Enterprise 3) while the
rest of AWIPS (and the PX) was running Linux RedHat 7.2. The software
was compiled for RedHat 7.2. Significant performance problems were noted
with the GHG software when running on the DX machine due to an
incompatibility with the Enterprise 3 operating system libraries.

***Mitigation Action:*** A solution to the DX failover problem has been
found and will be implemented with the next AWIPS software release. A
Redhat Enterprise 3 version of the software has been provided to the
field to use on DX systems.

## Training Provided

Training materials were developed by Shannon White (OS6) and Matt Davis
(ITO, WFO ARX) in video format (See Table 3). These short movies focused
on an aspect of the software training and provided a stand alone means
for forecasters to receive quality training. To further explain this
approach, the forecaster (perhaps on shift) would set aside 15 minutes
to go through one or more videos. She/he would insert the training disc
into an available PC and run the video. On the screen, the forecaster
would see the software interface and listen to a trainer explain what he
or she is doing on the screen. As the trainer explains, the forecaster
can see the mouse move, selections made and the software respond just
like it would do if he/she were controlling the software. The videos
provided one-on-one instruction, walking the forecaster through the use
of the software, without tying up the SOO or Focal Point. It provided
quality training that can be received on a flexible schedule.

In addition to the video training, the GHG software was made available
for use on the Weather Event Simulator (WES). The WES is the primary
training tool available to the WFO Science and Operations Officers (SOO)
to simulate forecast and warning operations in the office. The offices

were instructed to install GHG on the WES and to have all forecasters
practice using GHG. Using the WES, forecasters can safely (without
affecting office operations or transmitting products) practice

  --------------------------------------------------- -------------------
  **Video Training Movie (avi format)**               **Movie Run Time
                                                      (Min:Sec)**

  GFE Operating Modes                                 11:13

  IFPS 17.3 Product Editor Changes                    14:32

  Separate, Make and Merge Hazard Tools               12:58

  Simple Hazard Creation and Follow-up                10:49

  Complex Upgrade Scenario                            11:50

  Downgrade, Cancellation and Valid Time Changes      10:10

  Hazard Recovery                                     3:13

  Doing a Correction or Retransmission                5:45

  GHG Hazards Monitor                                 14:42

  WBC Operations                                      14:55
  --------------------------------------------------- -------------------

Table 3: Training videos produced and provided to GHG/VTEC OT&E sites on
the use of GHG software. The videos were short and focused on a specific
training need.

the generation of hazard grids and the production of watch, warning and
advisory products. At many OT&E sites, the SOO and/or office POC
provided one-on-one training using the WES to reinforce the video
training.

Also provided for use on the WES computer was a SPC watch simulator.
This program (written by Matt Davis) simulates the receipt of a proposed
watch (WCL), which is ingested into GHG. The watch simulator software
allows forecasters to see the entire Watch By County (WBC) process using
GHG so they can practice without the need for SPC to issue additional
test watch products. This provides an effective means for ensuring
proficiency for all forecasters at the office in the issuance and
updating of SPC watches.

The GHG test site Point of Contact (POC) was required to verify that all
forecasters had completed the provided training materials as a part of
the VTEC turn-on checklist.

### Training evaluation criteria

Training provided was assessed by surveying the OT&E participants that
received the VTEC and GHG training. The training evaluation will be used
to improve training to be provided to all forecasters for national GHG
and VTEC implementation. Training effectiveness will be evaluated at the
end of the OT&E.

[Success criteria]{.underline}: Survey responses from forecasters rating
effectiveness of various aspects of training to enable them to issue
VTEC using GHG. Success: greater than or equal to an average of 3 on
scale of 1-5.

[Failure]{.underline}: Survey responses indicate an average of less than
3 rating on scale of 1-5. Failure results in providing significant
improvement to training prior to offering training prior to VTEC going
operational.

### Training Survey Results

We received 164 responses to the training survey. Of those responding,
81% were satisfied with the training provided. On a scale of 1 to 5, the
training responses averaged 4.02. When asked about the training videos
provided, 80% liked having training delivered this way. Forecasters were
also provided the opportunity to comment on the training. Several
responses indicated that they needed more hands-on exercises for the
training to really sink in. Other comments reflected the need to provide
more time for this training. (OT&E sites were only provided a few weeks
to complete the training prior to the start of the OT&E).

Based on the responses of the survey, the training provided was
considered sufficient for the use of the software. Training videos are
being updated by OCWWS to include lessons learned in the OT&E.

## GHG Concept of Operations

The concept of operations evaluation examines the process of issuing
watch, warning and advisory products and the effect of the process on
the forecaster's ability to successfully produce and transmit the
product. A user survey was developed to solicit feedback on the
usability and workload associated with the use of GHG software as a part
of this process.

We received 280 responses from the survey. The responses were mainly
from regular rotation forecasters (80%). More than two-thirds (69%) had
7 or more years of forecasting experience while nearly all respondents
(97%) had at least 1 year of experience with the GFE software.

Nearly three-quarters of survey responses (73%) indicated that GHG had a
positive impact in helping them do their job. An additional 20 %
indicated that it had no discernable impact.

When asked about workload issues, a majority (63%) indicated that GHG
provided a decrease or no change in forecaster workload. Forecasters
that indicated that there was an increase sited several reasons. One
reason was that forecasters were required to issue both the experimental
WCN product and the official SPS product to communicate changes to
severe thunderstorm or tornado watches. Additional comments sighted the
need to improve product formatters for the CWF product. The forecast
text produced by the formatter is often edited. However, GHG and VTEC
cannot allow the forecaster to re-issue the forecast without re-running
the formatter, which then results in additional manual editing of the
forecast text.

# Issues and Recommendations

## Watch-By-County Issues

### Product Errors

The forecasters were able to produce the Watch County Notification (WCN)
message correctly 95.6% of the time. Nearly all of the WCN errors were
due to minor software bugs related to the wording of watch replacement.
Most of the failed products were missing the watch type words of the
watch that was being replaced. This has been corrected in the IFPS 17.5
version of the software.

### Extending Watches

In some circumstances, there is a need to briefly extend the watch for
an hour or 2 for a few counties instead of issuing a new watch. This is
allowed through close coordination with the Storm Prediction Center
(SPC). On one occasion, a product failure occurred when the forecaster
attempted to extend the watch, but the product was transmitted just
after the initial issuance of the watch had expired. The software then
treated this as an expiration (EXP) and new (NEW) coding using the same
ETN number in the product instead of an extension in time (EXT). Due to
the timing of the release of the product, the software followed the
rules, which allows the issuance of an expiration statement after the
watch has expired, but also because the watch was extended in the grids,
"NEW" coding was provided.

To avoid this problem, the concept of operations for extending watches
needs to have a cut-off time (say 15 minutes prior to the end of a
watch). Any extensions to the watch after this cut-off time would
require the issuance of a new watch. If the coordination is completed
too close to the end of a watch, then there may not be sufficient time
for the forecaster to alter the hazard grid and generate the new WCN
product for extending the watch prior to the expiration of the existing
watch.

### WOU Update Product

Testing of the WOU update process was conducted internally during the
OT&E. The WOU update is an automated process where updates to the
watches in effect are created from the WCN products received from field
offices. A WCN is issued by the WFO to communicate changes to the watch
(e.g., clear counties out of the watch). The local changes to the watch
as the severe weather event unfolds are then consolidated into an update
product which will be released each hour.

Since only 35 offices were producing WCN products during the OT&E, the
WOU update could not be generated and transmitted as an experimental
product. The WOU update can begin as soon as we have nearly all offices
using GHG and generating WCN products.

### Recommendation: Proceed with the national deployment of GHG with a focus on generating WCN products as quickly as possible. This will provide an opportunity to run test scenarios that exercise the WOU Update process with experimental VTEC coding.

## Marine Weather Statement Issue

The Marine Weather Statement (MWS) has three uses for the NWS. The
product is used to provide follow-up information concerning a Special
Marine Warning. This flavor of the MWS product currently contains
operational VTEC coding when used for this purpose with the MA.W
phenomenon coding. The MWS can also be used for describing longer fused
marine events and for communicating information about short-fused,
non-severe convective activity -- both using the MA.S VTEC coding.

Our attempts to use GHG for the short-fused, non-severe convective
activity in the OT&E immediately became problematic. Forecasters
routinely use the WARNGEN AWIPS application for these short-fused events
because this application provides the now-casting tools they need to
track and forecast the movement of these storms. Since GHG is designed
to work for long-fused watch, warning and advisories, it operates on a
time resolution of one hour. GHG also does not provide now-casting
tools. Therefore it does not make sense to require forecasters to use
GHG for these short-fused events. The short-fused MWS product instead
needs to be produced using the AWIPS WARNGEN program. However, WARNGEN
does not provide the full VTEC follow-up capabilities for the MWS
product in the current AWIPS software releases (OB4.2 or OB5.0).

The MWS was validated for long-fused marine events using GHG. Three
errors did occur during the OT&E, but these errors were not generated
from the use of GHG. These errors only occurred because one office had
altered WARNGEN to produce operational VTEC for MA.S short-fused
phenomena.

### Recommendation: Operational VTEC coding for the MA.S phenomena code in the MWS should be deferred and validated with the H-VTEC OT&E scheduled for this fall.

## Software Stability Issues

Offices reported instability with GHG software with severe IFPS server
crashes and lockups occurring at the WFOs at RNK and ILM. Also, an
increase in crashes was observed at OT&E sites after the office switched
to using the DX hardware (with AWIPS version OB4.2).

FSL analyzed the reason for the instability and found that it was caused
by an incompatibility between the software and the operating system on
the DX server. The DX server runs the RedHat Enterprise 3 (RHE3) Linux
operating system. The IFPS software was compiled for the RedHat 7.2
Linux operating system since this was the baseline operating system on
AWIPS servers prior to switching to the DX server. The software has been
recompiled for RHE3 for IFPS 17.5 and later versions.

## Service Backup Issues

Service backup was performed on numerous occasions during the OT&E due
to software upgrades or backup test case scenarios. The software was
designed to provide two types of service backup -- "full backup" and
"quick backup".

The full service backup for IFPS requires an office to download
configuration files and forecast grids from the centralized backup
server. The initiation of this backup capability can take some time to
complete during busy network periods due to the transfer of a large
amount of data. The quick backup method allows the office to switch into
backup mode without downloading data from the backup server. The
software allows you to generate warning products quickly for another
office using local configuration data.

Quick backup was not functional during most of the OT&E due to software
problems. A fix for the problem came late -- so limited testing of this
backup mode was completed. Several test case scenarios were completed
during the last week of the OT&E. These tests indicated that the
implementation of the quick backup method was faulty in some situations.
Based on the FFA product test failure in week 7 (mentioned in section
3), the quick backup method has been changed to download the
configuration files only from the centralized backup server, so the
office will have the latest local configuration files for the office
they are performing service backup for.

Offices that performed full service backup were able to successfully
provide backup, but software performance was poor. This was due to a
memory leak in the IFPS server software. This memory leak has been fixed
in IFPS 17.5.

### Recommendation: Conduct additional service backup testing with later versions of IFPS (17.5 or later) software to ensure reliable transfer of services in case of site failure.

## Headline Formulation Issues

### Requirements

Clearly defined requirements are essential to the software development
process. The locking of headlines to ensure the standardization of
headlines that reflect the VTEC coding was a strong requirement for GHG
software development. However, ideas and concepts from program leaders
don't always translate well into operations. This was the case with the
headline phrasing leading up to and during the OT&E which resulted in
multiple iterations of the details concerning the headline requirement.

### Issues

The main issue surrounded the use of timing phrases in the headlines.
The typical model was to use specific times in the short term and "fuzzy
times" (e.g., this afternoon or Thursday morning) for longer term
periods. These fuzzy times used terminology based on dividing the day up
into 4 six hour periods. The formulation would use the day or the week
(Wednesday) or "this" (or "to" for tonight) in front of the 6-hourly
descriptor (night, morning, afternoon, or evening).

In certain situations, we discovered headline formulation that was not
adequately communicating the intended timing of the event. Under some
instances, a forecaster would intend to upgrade an event, but the
product contained a headline indicating the two events were in effect at
the same time due to the 6 hour resolution of the fuzzy wording. For
example:

...SMALL CRAFT ADVISORY IN EFFECT FOR THURSDAY AFTERNOON...

...GALE WARNING IN EFFECT FOR THURSDAY AFTERNOON...

Since marine zones can be very large, the Regional Focal Points
requested that no specific times be used in the CWF marine product
headlines. This requirement change resulted from the IOT&E testing. This
change further confused the intent of some headlines by expanding the
fuzzy wording into the short term.

Another complication arose from the use of certain words (i.e., "UNTIL"
or "TO"). Events ending late in a period were indicated as being in
effect until that period. For example, for an event ending at 10 am
local time:

...SMALL CRAFT ADVISORY IN EFFECT UNTIL THIS MORNING...

Program leaders at NWS Headquarters, Regional Focal Points and NWS
partners came to an agreement for new specifications for the headline
phrasing. These requirements are specified in the Appendix A of NWSI
10-1701 and will be implemented in 17.6 and later version of IFPS
software.

## Additional Problems Identified by OT&E Testing

Two problems were identified that were not related to the OT&E, but OT&E
testing brought them to our attention. These issues were slowdowns in
the transmission products (including high priority warning products)
from a WFO on the AWIPS WAN and the occasional garbling of text products
on the National Weather Wire Service (NWWS).

The AWIPS WAN slowdowns were traced to the use of the Message Handling
System (MHS) software in AWIPS. Although the MHS system sends products
based on product priority, the efficient use of the MHS requires file
sizes to be small. The MHS was intended for the handling of NWS text
products. With time, new and larger data files have been placed into the
MHS. These large files can effectively block the sending of all products
from an office. The main source of these large files is the uploading of
forecast grids to the backup server. These grids are wrapped up into a
large compressed file and transmitted through the MHS. Also, grid files
are shared between offices for inter-site coordination (ISC) of the
forecasts. These ISC grids are smaller, but since they are sent to
multiple offices, the MHS transmits multiple copies on the WAN.

Since this problem was discovered, NWS offices have taken steps to limit
the amount and frequency of the grids that are sent through the MHS. The
NWS is also examining the implementation of "rsync" capabilities on
AWIPS to efficiently share updates to these files.

Garbled products came to our attention as reported errors with product
formats from one of our partners (Marvin McInnis at First Alert).
Occasionally a product will be truncated or incomplete and get merged
under the heading of another product. NWS is working with the NWWS
vendor to capture the problem, find the cause and correct it.

# Summary and Conclusions

The results presented in this report indicate that the GHG software is a
viable tool for the implementation of VTEC in all products evaluated.
The recommendations from this report to the NWS Corporate Board are:

-   Issue the Service Change Notification to produce Operational VTEC
    coding in beginning 11/1/2005 in the following products: WCN, FFA,
    CFW, CWF, NSH, NPW, WSW, RFW and WOU Update product

-   Delay operational VTEC implementation for MWS (MA.S phenomena code)
    until WARNGEN can produce required VTEC coding for the short-fused
    convective situations in the MWS product

-   Deploy IFPS 17.5 GHG software nationally as soon as possible to
    complete WBC process

-   Conduct additional service backup testing with IFPS 17.5 software

Along with these recommendations, the mitigation actions described in
section 2 have been (or soon will be) implemented to ensure a successful
implementation NWS wide.

# Appendix A: OCONUS Testing

## NWS Alaska Region GHG/VTEC OT&E Summary 

Although the Alaska Region is not required to implement VTEC in the fall
with the CONUS offices, they participated in the OT&E to get feedback on
how the GHG software would perform with Alaska's diverse climatological
regimes, large zones, multiple collectives for numerous products, and
forecaster workload. The creation of VTEC codes in Alaska products
performed flawlessly through the entire OT&E. However some issues need
to be addressed prior to implementation and will be mentioned here.

### Product Evaluation 

A total of 385 products were evaluated during the OT&E from the Alaska
Region, all from the Fairbanks (AFG) office. Some products were not
captured for evaluation in the product database (RFW products from the
WCZ collective and CWF products during the header change) and were not
evaluated. Of the 385 products evaluated, 4 failed the evaluation (see
table 4 for details).

  ------------------------- ----------------------- ----------------------
  Product Category          Total Evaluated         Number Failed

  CWF                       118                     2

  NPW                       144                     1

  RFW                       78                      1

  WSW                       45                      0

                                                    

  Total                     385                     4
  ------------------------- ----------------------- ----------------------

Table 4: Number of Alaska Region products evaluated and failed by
product category

### Failure Analysis 

##### Coastal Waters Forecast (CWF)

One failure was related to generating a product that had a small craft
advisory in effect from the issuance time, until 4pm, which resulted in
a "..." headline. Only the ellipse appeared without the worded headline.
This was a software bug which has been corrected in IFPS 17.5. Part of
the first failure, and all of the second was related to a forecaster
editing outside the GHG environment, which also introduced errors. Event
phenomena issued in the CWF product included small craft advisory, brisk
wind advisory, heavy freezing spray warning and gale warning.\
\
***Non-Precipitation Weather message (NPW)***

The one error for this product was due to a forecaster editing outside
the GHG environment, which introduced errors. Event phenomena issued in
the NPW product included wind advisory, dense fog advisory, dense smoke
advisory, high wind watch and high wind warning. WFO AFG also completed
a test scenario for extreme cold watch/warning, and a complex high wind
event.\
\
***Red Flag Warning (RFW)***

One failure was observed where an extension in time (EXT) code appeared
for an already expired segment. It appears that this was related to a
timing issue with the sending of the product. Event phenomena issued in
the RFW product included fire weather watch and red flag warning.\
\
***Winter Weather Message (WSW):\
***No Failures. Event phenomena issued in the WSW product was for
freezing rain advisory. AFG also completed a blizzard watch/warning
scenario.

### Comments and Conclusions

The GHG software was well received by most of the forecasters, and is
seen to be an important tool for issuing and monitoring warnings and for
the production of products with correct formats and headlines.

##### Workload Issues

The Alaska Region does not issue NPW or WSW products for advisories
since Alaska weather conditions would result in the production of a
large number of these products. The procedure is to add the advisory
headline to the appropriate zone(s) in the Zone Forecast Product (ZFP).
With the use of VTEC and GHG, forecasters will be required to issue the
advisory product (NPW or WSW). This added a significant workload for the
forecaster during the OT&E. To prepare for eventual VTEC implementation,
the Alaska Region offices are reviewing their advisory programs with
their partners to find efficiencies in the program to meet the service
need and reduce the forecaster workload

##### GHG Performance

Forecasters also commented about the performance of GHG. The system
performed slow and may be related to other AWIPS issues. Product
preparation time increased significantly due to the system slowdowns. A
few cases were reported where it took up to one-half hour to simply
produce and send an expiration statement with the associated adjustments
in the Zone Forecast Product (ZFP). Until system performance improves to
an acceptable level, the region will not be able to fully realize the
advantages of using the GHG and VTEC.

##### Multiple Domain/Multiple Collective Issues

Alaska Region offices issue multiple collectives for some products (ZFP,
WSW, NPW and CWF). Also, due to the size of the area of responsibility
at the Anchorage forecast office, the area is split into two separate
domains in IFPS. Testing of the Anchorage WFO configuration was
conducted using the Alaska Region Headquarters AWIPS system. The
software considers this split as two separate WFOs instead of one WFO.
This resulted in some interesting issues being discovered. The GHG is
designed to keep track of the VTEC ETN as one office (PAFG). With a
little work from FSL, GHG was configured to track the ETN numbers across
the collectives and multiple domains. However, additional issues
occurred when multiple forecasters were working with a specific
phenomena and product type at the same time for different collectives.
For example, at WFO Fairbanks, the forecaster issuing the WCZ product
interfered with the person issuing the AFG product, causing the AFG
forecaster to rerun the formatters. If you combine this situation with
slow software performance, the result is an unacceptable work
environment. A possible solution for the implementation of VTEC in
Alaska would be to treat the multiple domains the same as separate WFOs
-- with separate event tracking.

##### Headline Timing Phrases

The headlines in the CWF product became an issue during the OT&E. The
very large marine zone areas preclude the use of specific times for the
starting and ending of significant events. Updated policy resulted in a
different headline implementation in the OCONUS. The OCONUS will utilize
"fuzzy" time phrasing for all time periods in the CWF.

#####  Need for multiple events in a zone

Alaska Region zones are large in horizontal area and vertical elevation
change. It is common to have a watch in one portion of a zone and a
warning in another portion (or an advisory for lower elevations and a
warning for higher elevations). This scenario is also common in the
Western Region. Current NWS policy would dictate the occurrence of
either a watch or a warning within a zone for the same phenomenon.

Tests were conducted with the GHG software during the OT&E to allow
multiple VTEC strings for the same phenomena in the same zones and it
appears that the software could handle this new capability. It appears
that VTEC worked properly and the headlines could be edited to reflect
the conditions. This capability is required prior to implementing VTEC
in the Alaska Region.

### Concluding remarks 

The issues described here (along with issues concerning the watch,
warning and advisory program at Alaska Weather Service Offices) present
challenges to the implementation to VTEC in the Alaska Region. Since GHG
is the path for the implementation of VTEC, enough issues with the use
of GHG still need to be resolved before VTEC can become operational in
all Alaska products.

# Appendix B: Sample Test Case Scenario

![](media/image6.png){width="6.789583333333334in"
height="3.8055555555555554in"}

![](media/image7.png){width="6.795833333333333in"
height="3.4652777777777777in"}

![](media/image8.png){width="6.794444444444444in" height="3.16875in"}

![](media/image9.png){width="6.794444444444444in"
height="4.105555555555555in"}

![](media/image10.png){width="6.794444444444444in"
height="3.5444444444444443in"}

![](media/image11.png){width="6.795138888888889in" height="2.65625in"}

# Appendix C: Test Review Group

+-------------------+------+------------------------------------------+
| **Na              |      | **Function**                             |
| me/Organization** |      |                                          |
+===================+======+==========================================+
| Peter Browning    | CRx3 | Test Review Group Chair and OT&E Test    |
|                   |      | Director \[Voting Member\]               |
+-------------------+------+------------------------------------------+
| Deirdre Jones     | OST3 | WBC Project Manager                      |
+-------------------+------+------------------------------------------+
| Robert Rood       | O    | VTEC Project Manager \[Voting Member\]   |
|                   | ST32 |                                          |
+-------------------+------+------------------------------------------+
| Jerald Dinges     | O    | OPS Focal Point/Test Support             |
|                   | PS24 |                                          |
+-------------------+------+------------------------------------------+
| Herbert White     | OS51 | VTEC Dissemination Policy Focal Point    |
|                   |      |                                          |
| Art Kraus         |      |                                          |
+-------------------+------+------------------------------------------+
| Mark Tew          | OS22 | VTEC/WBC Services Program Manager        |
+-------------------+------+------------------------------------------+
| Mark Tew          | OS22 | Public and Fire Weather Services Focal   |
|                   |      | Point and Severe Weather Program Manager |
+-------------------+------+------------------------------------------+
| Tom Donaldson     | OS32 | Hydrologic Services Focal Point          |
+-------------------+------+------------------------------------------+
| Richard May       | OS21 | Marine Services Focal Point              |
|                   |      |                                          |
| Jamie Vavra       |      |                                          |
+-------------------+------+------------------------------------------+
| Shannon White     | OS6  | VTEC Training Focal Point                |
+-------------------+------+------------------------------------------+
| Russell Schneider | NCEP | NCEP (SPC) Focal Point                   |
+-------------------+------+------------------------------------------+
| Michelle Mainelli | NCEP | NCEP (TPC) Focal Point                   |
+-------------------+------+------------------------------------------+
| Laura Cutrer      | O    | Data Collection Focal Point              |
|                   | ST23 |                                          |
+-------------------+------+------------------------------------------+
| Greg Noonan       | C    | CRH Focal Point                          |
|                   | R3x1 |                                          |
| Jim Keeney        |      |                                          |
|                   | C    |                                          |
|                   | R1x1 |                                          |
+-------------------+------+------------------------------------------+
| Jason Franklin    | ER1  | ERH Focal Point \[Voting Member\]        |
|                   |      |                                          |
| Neal Dipasquale   |      |                                          |
+-------------------+------+------------------------------------------+
| Eric Howieson     | SR41 | SRH Focal Point \[Voting Member\]        |
|                   |      |                                          |
| Walt Zaleski      | SR1  |                                          |
+-------------------+------+------------------------------------------+
| Craig Schmidt     | WR1  | WRH Focal Point \[Voting Member\]        |
+-------------------+------+------------------------------------------+
| Duane Carpenter   | AR1  | ARH Focal Point \[Voting Member\]        |
+-------------------+------+------------------------------------------+
| Bill Ward         | PR11 | PRH Focal Point \[Voting Member\]        |
+-------------------+------+------------------------------------------+
| Marvin McInnis    | Par  | First Alert                              |
|                   | tner |                                          |
+-------------------+------+------------------------------------------+
| Kathy Strebe      | Par  | Weather Central                          |
|                   | tner |                                          |
+-------------------+------+------------------------------------------+

#  Appendix D: Offices Participating 

  ---------- ---------------------------------------------- --------------------
  ***\#***   ***Office***                                   ***NWS Region***

  1          WFO Pueblo CO (PUB)\*                          Central

  2          WFO La Crosse WI (ARX)\*                       Central

  3          WFO Des Moines IA (DMX)\*                      Central

  4          WFO Davenport IA (DVN)\*                       Central

  5          WFO Detroit, MI (DTX)\*                        Central

  6          WFO Cheyenne WY (CYS)                          Central

  7          WFO Springfield MO (SGF)                       Central

  8          WFO Minneapolis MN (MPX)                       Central

  9          WFO Chicago IL (LOT)                           Central

  10         WFO Norman OK (OUN)\*                          Southern

  11         WFO Tulsa OK (TSA)\*                           Southern

  12         WFO Huntsville, AL (HUN)\*                     Southern

  13         WFO Jackson, MS (JAN)\*                        Southern

  14         WFO Tampa Bay FL (TBW)                         Southern

  15         WFO Melbourne FL (MLB)                         Southern

  16         WFO Miami FL (MFL)                             Southern

  17         WFO Blacksburg VA (RNK)\*                      Eastern

  18         WFO Raleigh NC (RAH)\*                         Eastern

  19         WFO Charleston, SC (CHS)\*                     Eastern

  20         WFO Wakefield, VA (AKQ)                        Eastern

  21         WFO Greenville/Spartanburg (GSP)               Eastern

  22         WFO Morehead City, NC (MHX)                    Eastern

  23         WFO Wilmington, NC (ILM)                       Eastern

  24         WFO Columbia, SC (CAE)                         Eastern

  25         WFO Sterling, VA (LWX)                         Eastern

  26         WFO Billings MT (BYZ)\*                        Western

  27         WFO Glasgow MT (GGW)\*                         Western

  28         WFO Boise, ID (BOI)\*                          Western

  29         WFO Seattle, WA (SEW)                          Western

  30         WFO Great Falls, MT (TFX)                      Western

  31         WFO Portland, OR (PQR)                         Western

  32         WFO Spokane, WA (OTX)                          Western

  33         WFO Pendleton, OR (PDT)                        Western

  34         WFO Missoula, MT (MSO)                         Western

  35         WFO Fairbanks, AK (AFG)                        Alaska

  36         Alaska Region HQ (ARH)                         Alaska

  37         Pacific Region HQ (PRH)                        Pacific
  ---------- ---------------------------------------------- --------------------

# Appendix E: Acronyms

ARH Alaska Region Headquarters

AWIPS Advanced Weather Interactive Processing System

CONUS Continental United States

CRH Central Region Headquarters

CFW Coastal/Lakeshore Hazard Message

CWF Coastal Water Forecasts

ERH Eastern Region Headquarters

ETN Event Tracking Number

FFA Flood/Flash Flood Watch

GFE Graphical Forecast Editor

GHG Graphical Hazards Generation

IFPS Interactive Forecast Preparation System

ISC Inter-Site Coordination

ITO Information Technology Officer

MHS Message Handling System

MND Mass News Disseminator

MWS Marine Weather Statement

NCEP National Centers for Environmental Prediction

NCF AWIPS Network Control Facility

NOAA National Oceanic and Atmospheric Administration

NPW Non-Precipitation Weather Message

NSH Near Shore Forecast

NWS National Weather Service

NWSI NWS Instruction

NWWS National Weather Wire Service

OB Operational Build (for AWIPS)

OCONUS Outside Continental United States

OCWWS Office of Climate, Water, and Weather Services

OST Office of Science and Technology

OT&E Operational Test and Evaluation

POC Point of Contact

PRH Pacific Region Headquarters

RFW Fire Weather Watch/Warning Message

RHE3 RedHat Enterprise 3

SEC Systems Engineering Center

SMW Special Marine Warning Message

SOO Science and Operations Officer

SPC Storm Prediction Center

SRH Southern Region Headquarters

SVR Severe Thunderstorm Warning Message

SVS Severe Weather Statement

TOR Tornado Warning Message

TRG Test Review Group

TTR Test Trouble Report

VTEC Valid Time Event Code

UGC Universal Geographic Code

WAN Wide Area Network

WBC Watch By County

WCL Watch Coordination Message

WCN Watch County Notification

WES Weather Event Simulator

WFO Weather Forecast Office

WOU Watch Outline Update

WRH Western Region Headquarters

WSH National Weather Service Headquarters

WSW Winter Weather message

WWA Watch Warning and Advisories

Bottom of Form