![](media/image1.png)

See
[http://cdm.fly.faa.gov/Workgroups/weather_eval.html](http://cdm.fly.faa.gov/Workgroups/weather_eval.html )
for detail on all presentations. Weather Evaluation Team (WET) Work
Group (WG) convened April 21, 2008 at the St. Anthony Hotel in San
Antonio, TX.

***WET Meeting -- April 21***

***Introduction***

Danny Sims welcomed everyone and participants introduced themselves. The
meeting started with an overview of the Collaborative Decision Making
(CDM) Stakeholders Group tasking, which was broken down into two parts.

The first task was to look at 8-24 hour convective forecast products
that would be used for operations plan development and planning telcons.
The options outlined were the Storm Prediction Center (SPC) 12-24 hour
convective forecast products and an 8-hour CCFP. Both of these options
would extend beyond the current Collaborative Convective Forecast
Product (CCFP) forecast. The second task was to recommend a process to
present information on forecasted airport weather conditions to be used
in operations plan development and planning team telcons.

The purpose of this WG meeting was to initiate discussions and strategy
for these two tasks, and also review relevant work. Emphasis was placed
on obtaining the best possible forecast.

### *AGENDA*

a\) CCFP Comparison Study

b\) SPC Convective Outlook Study

c\) CCFP-LAMP MOS Combination

d\) Industry/ATCSCC/ARTCC use of TAFs

e\) Group Discussion and Strategy

f\) Next Steps

***a) CCFP Comparison Study***

Dave Simenauer presented a high level briefing of the results of the
Convective Product Comparison Study conducted by AvMet and Earth System
Research Laboratory (ESRL) during the summer of 2007. Forecasts of less
than 6 hours were eliminated from this discussion. Only the 6 hour CCFP
forecasts were included in the evaluation. A subjective comparison was
conducted for the following products:

-   CCFP Preliminary

-   CCFP Final - Used as Baseline

-   North American Meso (NAM) Simulated Radar Reflectivity (SRR)

-   Rapid Update Cycle (RUC) SRR

-   RUC Convective Probabilistic Forecast (RCPF)

The results of the survey are available at
<http://www.avmet.com/ccfp-interop/entry.cfm>. NAM and RUC reflectivity
products rarely forecast any convection to be above VIP level 2 and;
therefore, performed poorly. NAM and RUC determined structure of
convection fairly accurate in most cases. Even with the inaccuracy of
how the reflectivity products forecast intensity, adjustments could be
made to make these products more useful. Filtering out convection below
level 3 in the subjective evaluation of the reflectivity products led to
no significant activity being shown at all.

RCPF has finer granularity than CCFP and shows a better structure of
weather. Often times the correlation between RCPF and CCFP was very
close, and at times RCPF provided a better indication of the weather
structure than CCFP. RCPF appears to do better in the 6-8 hour range
than it does in the shorter forecasts. The group discussed the benefits
and issues with the possible collaboration of CCFP and RCPF.

***b) SPC Convective Outlook Study***

John Huhn presented information on the Storm Prediction Center (SPC)
convective outlooks. CCFP criteria requirements were discussed, with an
emphasis on how they may make CCFP seem inaccurate. Since CCFP provides
forecasts up to 6 hours, the CCFP is ineffective for strategic planning.
This is due to the fact that during the early morning Strategic Planning
Telcons (SPT), there is no CCFP information for afternoon convective
activity. Mr. Huhn's briefing presented information from several case
studies in 2007. These case studies compared SPC outlook to the CCFP
forecast. LAMP and CCFP forecasts were also compared to SPC outlook in
an April 2008 case study. The case studies indicate that there are
instances when the SPC outlook may provide advanced notice (\> 6 hours)
to widespread convective outbreaks.

Additionally, based on the assumption that ATM Decision makers access
the CCFP chat records:

-   An 8 hour anticipated preview could be added to 10:15-10:45Z and
    12:15-12:45Z chat session output.

It is proposed to add a few lines on the 8-hour convective outlook at
the end of the chat instead of a summary of the discussions.

***c) CCFP-LAMP MOS Combination***

Tom MacPhail stated that the meteorology behind CCFP is good, but CCFP
is not capturing all significant convective weather due to the
requirements that have been placed on CCFP and CCFP is fundamentally
flawed. The collaboration of CCFP is viewed as critical but does not add
significant value to the product. The enhanced CCFP requirements,
identified by the WET in the 2006 Recommendation document include:

-   year-round issuance,

-   expanded geographic coverage, and

-   an 8-hour forecast.

The drawback to enhancing the CCFP, from the National Weather Services
perspective, is that it would require the addition of 5 employees at
AWC. A proposal was made to change CCFP without impacting available
resources. This would involve combining the current CCFP with a
model-based probabilistic forecast (LAMP MOS is currently the best
choice for the automated portion). The WG went through the April 3^rd^,
2008 case study comparing LAMP MOS output to CCFP. 2, 6, 8, and 10 hour
LAMP MOS forecasts were compared to the 2-hour CCFP.

***d) Industry/ATCSCC/ARTCC use of TAFs***

Tom Fahey gave the WG a brief background on terminal area forecasts
(TAFs). Representatives of the industry, ATCSCC, and ARTCC gave the WG
an overview of their uses of TAFs.

Industry Perspective:

The SPT were not intended to resolve weather forecast issues. The
original agreement was that detailed discussions regarding weather
forecast issues would not be conducted during the SPT. From the industry
perspective, there have been many approaches over the past few years
that address weather specifically from the airport terminal perspective.
There is a need to further define how this information is provided to
the decision makers. Sometimes there has been a separate telcon
established to discuss weather conditions at a specific airport. These
separate telcons have been relatively successful; however, there isn't a
general understanding from industry about how TAF related information is
being input into the traffic management decisions. A process of
conveying TAF information to air traffic managers needs to be identified
and formalized.

ATCSCC Perspective:

The midnight shift by the National Traffic Management Officer (NTMO)
gathers a list of all TAFs provided by the National Weather Service at
4:30 am, reviews the list, identifies, and formulates the first plan of
the day. There usually are not many Traffic Management Initiatives
(TMIs) that result from these early morning TAFs. TAF updates are
continuously received and monitored. Feedback is provided by weather
specialists who participate in the chat room and gather information from
a variety of sources. TAFs help to determine when to utilize TMIs.

ARTCC Perspective:

In the Enroute TMU, TAFs are used for a variety of reasons. Factors that
are considered in TAFs are: ceilings, visibility, wind direction and
speed, and precipitation. TMUs assume that since a TAF is a forecast,
that it will have a certain percentage of accuracy. Some TMUs rely
heavily on a TAF, while other TMUs rely heavily on CWSU interpretation
of a TAF, or they create a "hybrid" using different sources of
information and interpretation. TAFs are helpful for Airport Acceptance
Rate (AAR) impacts, but not as helpful for convection.

***e) Group Discussion and Strategy***

Multiply times there were discussions stating the best strategy should
be to combine the "best of both worlds". This referred to utilizing both
automation and human forecaster input, part of that input being
collaboration. For automation, it was suggested that an 8-hour output be
used. Some members felt that the earliest projected timeframe for this
8-hour product is 2011. Subsequent discussions occurred regarding
creating a demonstration that actually takes the 8-hour LAMP MOS and
converts it to what would look like a CCFP, in which case some
conversion automation would need to be defined. The issue is that at
8-hours and beyond, the product would not "look" like a CCFP, and
currently everyone is accustomed to CCFP. The issue is in developing the
technology and making it look similar to CCFP. There is also a risk in
"dumbing it down" in trying to make the product look like the CCFP.

There were two proposed sources of forecast information to choose from:

-   LAMP MOS and/or

-   SPC products

If the group can decide what it wants tested, they can come back
together during the summer to look at the results once the testing has
been conducted.

The group agreed that a 12-hour product would be highly beneficial
because it would allow for a better strategy for routes, lay the
groundwork for better dialogue in planning telcons, remove inaccurate
predictions, and improve the management of special events.

There was concern that there is already enough discrepancy between
interpretations of what is on CCFP between FAA facilities. If we start
utilizing two products, it could cause even more discrepancy. The
response to this was that there should be only 1 product, but based on
output from two sources. Also, CCFP is proven technology, so it would
not be desirable to quit using it in the near future.

The group discussed issues regarding collaborative TAFs. Currently there
are times when it appears as if multiple people collaborate to discuss
an issue, but this is not a consistent practice. From an ATCSCC
perspective, if they wanted feedback at an airport such as one in
Chicago, they would collaborate with airline meteorological personnel
within that area.

It was suggested that there be one repository of all TAF information
available for each airport. Currently, several airlines develop a TAF
for a specific airport and submit to the ATCSCC, other airlines may be
submitting it to the ARTCC &/or CWSU facility.

It was proposed that a process be developed in which, if all TAFs from
various sources are within 10% of one other, they are consolidated into
one TAF for that airport.

It was agreed that it would be very beneficial to be able to see all
TAFs (NWS as well as private forecasting organizations and related CWSU
products for the airport. If a company decides to withhold information,
they should not be given any information in return. It was decided to
further investigate how to make this information available via a web
application.

***f) Next Steps***

The follow-up meeting is a telcon scheduled for **Tuesday May 6^th^,
2008 at 1:00 pm**.

***Appendix 1: Attendees***

  ---------------- ------------------ --------------- --------------------------
  **Name**         **Organization**   **Telephone**   **e-Mail**

  Kris Kimmons     Continental        713-324-7500    kkim@coair.com

  Jeff Sarver      UPS                502-359-7075    jsarver@ups.com

  Danny Sims       FAA                703-326-3938    danny.sims@faa.gov

  Jean Charest     MSC                514-283-9969    Jeanf.charest@ec.gc.ca

  John Huhn        MITRE              703-983-2050    jhuhn@mitre.org

  Mark Phaneuf     AvMet/FAA          703-351-5647    mphaneuf@avmet.com

  Tom MacPhail     NWS                703-925-3135    Thomas.mccphail@nasa.gov

  Mark Huberdean   Mitre/CAASD        703-983-5906    mwhuber@mitre.org

  David Simenauer  AvMet/FAA          703-351-5648    simenauer@avmet.com

  Tom Fahey        NWA                612-726-3256    tom.fahey@nwa.com

  Brad Sherman     FAA                703-904-4525    brad.sherman@faa.gov

  Beth Thomas      FAA                703-904-4525    beth.thomas@faa.gov

  Dean Fulmer      FAA                651-463-5505    dean.fulmer@faa.gov

  Steve Privott    TAC2               703-453-1308    steve.privott@auatac.com

  Brian Collins    Southwest Airlines 214-717-1621    brian.collins@wnco.com

  Dan Gilani       TAC2               703-326-3909    daniel.gilani@auatac.com
  ---------------- ------------------ --------------- --------------------------