# Presentation: 856762

## Verification &Weather Impacts:Past and Future

- Verification &Weather Impacts:Past and Future

- Mark Fox
- Regional Training Officer

## National Weather Service

- Currently Verify:
- TOR / SVR / FFW ( POD – FAR – CSI )
- Max / Min T  ( in various forms )
- PoP ( brier, reliability, and other forms )
- Marine Wind Speed / Wave Heights
- TAF (various forms)
- 50% Offices say their “Best Service” is Direct and Personal Communications

##  A Goal:

- Begin the culture change of “beating the model” to ‘Get it right when it matters.’
- Focus should be on:
- Impact Weather
- Accurate Grids
- How ?
- Change Habits
- Change perceptions of the models
- “Garbage that the model spits out”
- Change definition of success ?
- Change what we measure ?
- Reward what we do (even if it fails!) ?
- Good AFD with a “bad” MAE
- Or: Praise a “bad” forecast for the right reasons

## Weather Impacts

- The Customer Defines the Impact
- Not always what **w**e think the impact is
- We know the meteorology (and the limitations of the meteorology)
- SSDs Working Definition of *Impact Weather*:
- A weather event that causes a substantial departure from the normal routine.  The event must be forecastable with sufficient lead time to allow decision makers to take appropriate action that results in a saving of lives and/or a reduction in property damage or other adverse economic impacts.

## Forecasting for Weather Impacts

- SR Grid policy is designed to allow forecasters to concentrate on the weather that will create impacts
- WISE Concept came out in 2005
- SSD / CWWD survey in early 2006:
- 90% said our office are using the grid policy, but our neighbors are not.
- Idea behind WISE, or looking for “windows of opportunity” is to let the model do what it does best and allow the forecaster to do what he/she does best.

## WISE?   What about ConOps ?

- ConOps is discussing infrastructure (grid size, cluster, equipment, etc.), but not operations
- We have the opportunity to influence operations for the ConOps
- Already being done in many offices

## Southern Region Grid Policy

- Does the Model Make Meteorological Sense ?
- Yes : Use it
- No  : Don’t
- Everyone uses the same thing as the *starting point* to collaborate the forecast
- Concentrate on the Impact Events or where the model is off base
- Routine errors minimized

## Southern Region Grids - Before

## Southern Region Grids - After

## Why the Jump ?

- Regime ?
- Western drought
- Above normal winter
- Model MAE
- 4.2 in 2005 / 2006
- 4.1 in 2004 / 2005
- Man / Machine Mix ?
- Forecast more than 4 degrees from MOS
- 13 % of the time in 2005 – 2006
- 17 % of the time in 2004 - 2005

## A Possible Reason

## Southern Region Grids - Before

## Southern Region Grids - After

## Why the Jump ?

- Regime ?
- Western drought
- Above normal winter
- Model
- 0.11 Brier Score in 2005 / 2006
- 0.08 Brier Score in 2004 / 2005
- Man / Machine Mix ?
- Forecast more than 20 % from MOS
-    6 % of the time in 2005 – 2006
-  10 % of the time in 2004 - 2005

## A Possible Reason ?

## The Future ?

- But...wouldn’t grid PoP verification make more sense ??
- Jack.settelmaier@noaa.gov

## Severe Weather Warnings

- Public Forecast Verification Nice for Forecasters
- Not Tied to GPRA Goals
- Hazardous Weather Warnings are in the Bread and Butter of NWS
- Tied to GPRA Goals

## Tornado Warnings

## Tornado Warnings

## Making the Shift

- Get away from the routine
- Need a database instead of ‘Stats on Demand’
- Brent.macaloney@noaa.gov
- Verify the stuff that matters
- Grids instead of points (lots of issues here)
- BOI Verify
- Large temperature swings
- Critical Temperatures
- PoP timing
- Hazards and threats

## Making the Shift

- Verify the stuff that matters
- Mississippi Valley Tornado Outbreak
- DFW Snow / Sleet
- West Texas December Cold Snap
- Florida Fires
- Fire Weather Season in General
- Anything else I can’t think of...
- How many of us were told:
- Great job beating MOS !!!