NASA George C. Marshall Space Flight Center

Earth Science Applications Program

Disaster Management

Interim V/V and Benchmarking Summary

**Steven Goodman, PhD**

**Project Manager**

The Southern Thunder Project

**George C. Marshall Space Flight Center, AL**

**August 2005**

**ATTACHMENT D**

**Interim V/V and Benchmarking Summary from LMA surveys through May
2005**

There were a total of 23 surveys filled out by NWS forecasters from HUN
and OHX. (There are 25 unique survey IDs, but one of these was a
continuation of a previously uncompleted survey, while another was
submitted twice). There were two surveys filled out for different storms
on one day, while two different forecasters (from the same office) each
filled out a survey for the same storm period.

The first survey was filled out in November 2003 by the Huntsville
office. This was the only survey for 2003. In 2004, 8 surveys were
completed, all by the Huntsville office. So far in 2005, 14 surveys have
been filled out- 11 by HUN, 3 for OHX.

A number of different severe weather scenarios were represented in the
surveys: large hail, tornadoes, straight line winds, cells embedded
within lines, supercells, and microburst-producing summertime pop-up
thunderstorms.

Of the three OHX surveys (the office that has had LMA access for the
least time), they all indicated that the data were useful in determining
which cells had the strongest updrafts. This helped them to identify
which storms deserved more attention. The lightning data was used as a
verification of other data (i.e., radar, human based reports) utilized
by the forecasters in the warning decision. In one of the three surveys,
they did estimate that the warning decision was positively affected (by
2-3 min) by using the LMA data. It seems the OHX office has only
recently started to have confidence in the data, and are now starting to
use it more.

Of the 20 HUN surveys, there were 4 instances in which the LMA data were
not used in warning decisions because 1) in one case, the storm was too
far away for useful analysis \[survey 11\], 2) in another case,
forecasters thought the data looked 'flaky' \[a stationary noise source
was the probable cause of this, survey 3\], and 3) 2 cases where the LMA
data weren't properly ingested into AWIPS \[survey 2 and 10\]. In 8 of
the HUN surveys, the LMA data were estimated to have a positive impact
on the warning decision of between 1-7 minutes. In the other 8 cases,
there was either no impact on the warning decision or the forecaster
indicated that the LMA did not affect the warning decision lead time.

Note on BMX: We have not had a survey filled out by anyone in the BMX
office. In January, we heard for the first time that they were having
trouble getting the data. That issue appears to have been corrected. In
one instance this year when we contacted BMX about whether they used the
LMA during an event and they said they due to the timing of the event
(middle of the night/early morning) they didn\'t even think about the
LMA data. One issue that could be affecting their usage of the LMA data
is that only a portion (\~50% or so) of their warning area is covered by
the LMA data.

In 2003, only 1 survey was filled out as the survey had only recently
been implemented at the HUN office. In 2004, 8 surveys were filled out-
all by HUN. So far in 2005, 14 surveys have been completed- 11 by HUN
and 3 by OHX.

It should be noted that not all forecasters in a given office use the
LMA data. Some that do may not fill out the survey for all cases.
Sometimes the surveys are filled out 2 or more days after an event has
occurred. The main forecaster at the time may not have filled out the
survey, but someone else in the office did. Sometimes the survey was
filled out by someone else (SOO or other) after interviewing the
forecaster that was on duty during the event. Also, some surveys were
for entire episodes of multiple storms/warnings while others were for 1
only 1 storm. There are also some inconsistencies in the surveys, for
example survey 14 says that the LMA did not have an impact on the length
of time it improved the warning decision, but in the comments states
that "this allowed us additional (10-15 min) to issue/update Significant
Alerts for the affected county".

For the 19 cases where the LMA was available, the importance of various
parameters used by forecasters were:

Reflectivity signatures: 9.1

LMA 6.7

Near storm environment: 5.8

Human report: 5.2

Strong Rotation: 5.2

Boundaries: 3.8

NLDN: 3.7

TVS: 2.3

Previous severe wx: 1.5

Avg LMA warn. decision improvement: 1.5-2.0 minutes

Total \# warnings: 121

Total \# surveys: 19

\# cases LMA positively affected warning decision: 9

Of course, in non-tornadic situations, the TVS, strong rotation
signatures were of less importance than some other parameters. Human
reports, when available, were of great importance. The total lightning
(LMA) data were deemed twice as important as NLDN data, although the
forecasters could be biased since they were evaluating the LMA as a new
data source.

**[Severe Storms only surveys:]{.underline}**

These are results for episodes of severe warnings:

Reflectivity signatures: 8.7

LMA: 7.0

Near storm environment: 5.2

Human report: 4.1

NLDN: 3.7

Boundaries: 3.4

Strong rotation: 3.2

TVS: 1.5

Previous severe wx: 0.2

Avg LMA warning decision improvement: 1.8-2.4 minutes

Total \# warnings: 80

\# surveys 13

\# cases LMA positively affected warning decision: 8 (2.9-3.9 min avg
improvement)

**[Episodes with tornado warnings:]{.underline}**

Reflectivity signatures: 9.7

Strong rotation: 9.3

Human report: 7.7

Near storm environment: 6.8

LMA: 6.2

Boundaries: 4.5

Previous severe wx: 4.5

NLDN: 4.0

TVS: 3.8

Avg LMA warning decision improvement: 1-1.2 minutes

Total \# warnings: 41

\# surveys (with tornado warnings) 6

\# cases LMA positively affected warning decision: 1 (6-7 min
improvement)

It should be noted that if present, the TVS signature, strong rotation
and human-based reports of severe weather were considered of great
importance to the warning decision.

Overall, the LMA data ranked 2^nd^ in importance to yhe warning decision
to reflectivity signatures. For situations that had tornado warnings,
the LMA was considered the 5^th^ most important piece of information to
the warning decision. The LMA was 2^nd^ in importance in non-tornadic
episodes. There were more non-tornadic then tornadic situations in our
database.

Overall, the LMA data ranked 2^nd^ in importance to the warning decision
to reflectivity signatures. For tornado warnings only, the LMA was
considered the 5^th^ most important piece of information to the warning
decision. The LMA was 2^nd^ in importance in non-tornadic episodes.
There were more non-tornadic than tornadic situations.

**[Papers:]{.underline}**

Goodman, S. J., R. J. Blakeslee, H. J. Christian, W. J. Koshak, J.
Bailey, J. Hall, E. W. McCaul, Jr., D. Buechler, C. Darden, J. Burks, T.
Bradshaw, and P. Gatlin, 2005: The North Alabama Lightning Mapping
Array: Recent Severe Storm Observations and Future Prospects. *Atmos.
Res*., **76**, 423-437.

Darden, C, P. Gatlin, J. Burks, S. J. Goodman, D. Buechler, and J. Hall,
2006: Total Lightning in the Warning Decision Making Process -- Two
Years of Case Studies**,** Second Conference on Meteorological
Applications of Lightning Data, Amer. Met. Soc., Atlanta, CD-ROM.

Gatlin, Patrick, and S. Goodman, 2006: Total Lightning Signatures in
Tennessee Valley Thunderstorms. Second Symposium on Meteorological
Applications of Lightning Data, Amer. Met. Soc., Atlanta, CD-ROM.

Goodman, S. J., R. J. Blakeslee, D. J. Boccippio, H. J. Christian, W. J.
Koshak, and W. A. Petersen, 2006: GOES-R Lightning Mapper (GLM) Research
and Applications Risk Reduction. Second Symposium Toward a Global Earth
Observation System of Systems---Future National Operational
Environmental Satellite Systems, Amer. Met. Soc., Atlanta, CD-ROM.

Goodman, S. J. W. M. Lapenta, K. La Casse, E. McCaul, and W. Petersen,
2006: Storm Scale Forecasts and Observations of a North Alabama
Hailstorm on December 10, 2004. Symposium on the Challenges of Severe
Convective Storms, Amer. Met. Soc, Atlanta, CD-ROM.