# Presentation: 137128

- Event Probabilities
- Kathryn Gilbert (MDL/MOS)
- Judy E. Ghirardelli (MDL/LAMP)
- “Uncertainty is thus a fundamental characteristic of hydrometeorological prediction, and no forecast is complete without a description of its uncertainty. “
- NRC Report, “Completing the Forecast...,” 2006
- Meteorological Development Laboratory
- National Weather Service
- September 19, 2007

## Model Output Statistics (MOS)

- MOS provides objective forecast guidance for sensible weather elements
- Need historical record of observations
- Regression equations are applied to future runs of similar forecast model
- MOS post-processing ensures values are statistically consistent

## Model Output Statistics (MOS)

- MOS provides objective forecast guidance for sensible weather elements
- Need historical record of observations
- Regression equations are applied to future runs of similar forecast model
- MOS post-processing ensures values are statistically consistent
**Produces probability forecasts from a single run of NWP model**

## Statistical Post-Processing

- Advantages
-  Reflect the predictability of the event
-  Removal of some of the systematic model bias
-  Reliable probabilities
- unbiased over the entire range of forecasts
-  Specific elements and site forecasts
- Disadvantages
-  Short samples
-  Changing NWP models
-  Availability & quality of observations

## Explicit Probability Guidance

** ****KPHL   GFS MOS GUIDANCE    2/14/2007  0000 UTC                      **
** ****DT /FEB  14            /FEB  15                /FEB  16          /  **
** ****HR   06 09 12 15 18 21 00 03 06 09 12 15 18 21 00 03 06 09 12 18 00 **
** ****X/N                    32          15          26          13    29 **
** ****TMP  23 24 25 29 29 28 26 22 18 17 16 19 24 25 23 19 17 16 15 27 25 **
** ****DPT  20 23 25 29 28 23 16  8  6  4  3  2  0  0  0  2  3  4  4  5  5 **
** ****CLD  OV OV OV OV OV OV OV OV FW CL SC SC SC BK BK BK SC SC SC BK CL **
** ****WDR  06 06 05 02 34 31 30 30 29 29 28 29 29 28 28 28 27 27 27 28 27 **
** ****WSP  15 16 19 17 19 24 25 24 21 18 15 19 20 20 14 13 13 13 13 20 15**
-  **P06       100    84    32     0     0     0     0     0     0  0  0 **
** ****P12                    85           2           0           0     0**** **
** ****Q06         5     5     1     0     0     0     0     0     0  0  0 **
** ****Q12                     4           0           0           0     0**
-  **T06      3/ 0  1/24  2/ 0  0/ 0  1/ 1  1/19  0/ 6  0/ 3  0/ 0  0/ 0 **
** ****T12            3/24        2/26        1/19        1/23     0/21    **
-  **POZ  69 61 50 36 19 20 20  7  1  1  0  0  0  3  0  0  0  0  0  3  0 **
** ****POS  16 12 10 25 30 45 64 76 84 99 85 97 97 86 88 85 81100 93 91 94**** **
** ****TYP   Z  Z  Z  Z  S  S  S  S  S  S  S  S  S  S  S  S  S  S  S  S  S **
** ****SNW                                 1                       0       **
** ****CIG   4  3  3  3  3  4  6  6  6  8  8  8  8  8  8  8  8  8  8  8  8 **
** ****VIS   7  5  4  3  5  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7 **
** ****OBV   N BR BR BR BR  N  N  N  N  N  N  N  N  N  N  N  N  N  N  N  N**

## Hidden Probability Guidance

** ****KPHL   GFS MOS GUIDANCE    2/14/2007  0000 UTC                      **
** ****DT /FEB  14            /FEB  15                /FEB  16          /  **
** ****HR   06 09 12 15 18 21 00 03 06 09 12 15 18 21 00 03 06 09 12 18 00 **
** ****X/N                    32          15          26          13    29 **
** ****TMP  23 24 25 29 29 28 26 22 18 17 16 19 24 25 23 19 17 16 15 27 25 **
** ****DPT  20 23 25 29 28 23 16  8  6  4  3  2  0  0  0  2  3  4  4  5  5**
** ****CLD  OV OV OV OV OV OV OV OV FW CL SC SC SC BK BK BK SC SC SC BK CL**
** ****WDR  06 06 05 02 34 31 30 30 29 29 28 29 29 28 28 28 27 27 27 28 27 **
** ****WSP  15 16 19 17 19 24 25 24 21 18 15 19 20 20 14 13 13 13 13 20 15**
** ****P06       100    84    32     0     0     0     0     0     0  0  0 **
** ****P12                    85           2           0           0     0**** **
** ****Q06         5     5     1     0     0     0     0     0     0  0  0 **
** ****Q12                     4           0           0           0     0**** **
** ****T06      3/ 0  1/24  2/ 0  0/ 0  1/ 1  1/19  0/ 6  0/ 3  0/ 0  0/ 0 **
** ****T12            3/24        2/26        1/19        1/23     0/21    **
** ****POZ  69 61 50 36 19 20 20  7  1  1  0  0  0  3  0  0  0  0  0  3  0 **
** ****POS  16 12 10 25 30 45 64 76 84 99 85 97 97 86 88 85 81100 93 91 94**** **
** ****TYP   Z  Z  Z  Z  S  S  S  S  S  S  S  S  S  S  S  S  S  S  S  S  S **
** ****SNW                                 1                       0       **
** ****CIG   4  3  3  3  3  4  6  6  6  8  8  8  8  8  8  8  8  8  8  8  8 **
** ****VIS   7  5  4  3  5  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7 **
** ****OBV   N BR BR BR BR  N  N  N  N  N  N  N  N  N  N  N  N  N  N  N  N**

## Available MOS Probability Guidance

| Probability  | Event |
| --- | --- |
| 6-, 12-, 24-h Prob. Of Precipitation
Liquid equiv. ≥ 0.01”  | Yes/No |
| 3-h 20-km Thunderstorm
1 or more CG lightning strikes in grid box | Yes/No |
| 3-, 6-, 12-, 24-h 40-km Thunderstorm
1 or more CG lightning strikes in grid box | Yes/No |
| 6-, 12-, 24-h 40-km Severe Thunderstorm
(conditional on thunderstorms)
(unconditional = tstm*csvr)
(hail, damaging winds, tornado) | Yes/No |
| 1-, 3-h Precipitation occurring on the hour | Yes/No |

## Available MOS Probability Guidance

| Probability  | Event |
| --- | --- |
| 6-h Quantitative Precipitation Forecast | 0.01”, 0.1”, 0.25”, 0.5”, 1” + |
| 12-, 24-h Quantitative Precipitation Forecast | 0.01”, 0.1”, 0.25”, 0.5”, 1”,
2” + |
| Precipitation type 
(Conditional on Precipitation) | Freezing, Frozen, Liquid |
| 24-h Snow Amount* | >T, 2”+, 4”+, 6”+, 8”+ |
| Precipitation Characteristics
(Conditional on Precipitation) | Drizzle, Continuous, Showers |

- * Conditioned on precip occurrence and precip being frozen

## Available MOS Probability Guidance

| Probability  | Event |
| --- | --- |
| Ceiling (feet) | <200, 200-400, 500-900, 1000-1900, 2000-3000, 
3100-6500, 6600-12000, >12000 or unlimited) |
| Visibility (mi) | < 1⁄2, <1, <2, <3, ≤5, ≤6, >6  |
| Obstruction to Vision | None, Haze/Smoke, Mist, Fog, Blowing Phenomena |
| Total Sky Cover 
(Octets) | Clear 0/8, Few 1/8-2/8, 
Sct 3/8-4/8, Bkn 5/8-7/8,
Ovc 8/8 |

## Probability Forecast Considerations

-   Point Probabilities
-   METAR-based PoPs, QPF, Precip Type, Sky Cover...
-   Areal Probabilities
- 	remotely sensed observations (lightning) - gridded T-storms

**MOS forecasts can be:**

## Probability Forecast Considerations

-   Point Probabilities
-   METAR-based PoPs, QPF, Precip Type, Sky Cover...
-   Areal Probabilities
- 	remotely sensed observations (lightning) - gridded T-storms

- Assessment of probability is ***EXTREMELY**** *dependent upon how predictand “event” is defined:

**Something to keep in mind:**

-  Time period of consideration
-  Area of occurrence

**MOS forecasts can be:**

## Areal Probabilities 

*Time period?*
**    ****1 hour**
**	****  ****2 hours**
**	****  ****3 hours**
**	****  ****6 hours**
**	**** ****12 hours**
*Grid size?*
**   ****2.5 km**
**   ****10  km**
**   ****20  km**
**   ****40  km**
**   ****95  km**

- What is “appropriate” for thunderstorms?

## 40-km contoured 10%

**40-km contoured 10%**

**20-km contoured 10%**

- Probability of 1 or more cloud-to-ground lightning strikes in a grid box over a 3-h period
- Identical techniques, different horizontal resolutions

## 40-km contoured 10%

**40-km contoured 10%**

**20-km contoured 5%**

- Magnitude of the probabilities is proportional to the grid spacing, similar behavior with length of time periods

## Conversion of Probability Forecasts 

-      The MOS system ...produces probability forecasts for discrete precipitation amount classes.  The publicly issued precipitation amount forecasts were then derived by converting the underlying probabilities to the nonprobabilistic format by choosing one and only one of the possible categories.  ** **
-                             .
*from *_*Statistical Methods in Atmospheric Sciences*_*, 2006, Daniel S. Wilks*

## MOS Best Category Selection

- QPF Probability Example

- NO

**PROBABILITY (%)**

- NO

- NO

- NO

- 4

- 1

- 6

- 3

- 2

- 5

- 0

- YES

- YES

**THRESHOLD**

**EXCEEDED?**

**TO MOS GUIDANCE MESSAGES**

## Example of the Best Category Selection

**Threshold value ~ 16%**

**Probabilistic**

**Categorical**

## Conversion of Probability Forecasts 

-      The MOS system ...produces probability forecasts for discrete precipitation amount classes.  The publicly issued precipitation amount forecasts were then derived by converting the underlying probabilities to the nonprobabilistic format by choosing one and only one of the possible categories.  ** **
-                             .
*from *_*Statistical Methods in Atmospheric Sciences*_*, 2006, Daniel S. Wilks*

## Conversion of Probability Forecasts 

-      The MOS system ...produces probability forecasts for discrete precipitation amount classes.  The publicly issued precipitation amount forecasts were then derived by converting the underlying probabilities to the nonprobabilistic format by choosing one and only one of the possible categories. **This unfortunate procedure is practiced with distressing frequency, and advocated under the rationale that nonprobabilistic forecasts are easier to understand.  However, the conversion from probabilities inevitably results in a loss of information, to the detriment of the users of the forecasts**.
-                             .
*from *_*Statistical Methods in Atmospheric Sciences*_*, 2006, Daniel S. Wilks*

## Example of the “Unfortunate Procedure”

**Threshold value ~ 16%**

**Probabilistic**

**Categorical**

## Sample Products, etc...

## Graphics Products Available from weather.gov

## Slide 23

## http://weather.gov/mdl/synop/gridded/sectors

## Slide 25

## Slide 26

## Slide 27

## Slide 28

## Slide 29

## Slide 30

## Slide 31

## Slide 32

## Slide 33

## Slide 34

## Slide 35

## Slide 36

## Forecast Performance 

- almost finished

- Reliability Diagram
- 	how well are we calibrated?
- Brier Score
- 	a measure of accuracy,
- 	mean squared error of the forecasts

## Probability of Quantitative Precip ≥ .25” 

## Brier Score, Warm Season, GFS MOS 12-h PoP

- Independent data, July – October 2005, Sept 2006

## Available MOS Probability Products

- Alphanumeric Bulletins
- PoP, Thunderstorm, C SevereTstm, Snow, Freezing
-     Station guidance in BUFR format
- Contains all MOS probabilities
- Gridded MOS guidance in graphical and GRIB2 format (NDGD)
- 3-, 6- and 12-h thunderstorm probabilities
- 6- and 12-h probability of precipitation
- Web graphics of most MOS probability products
- _http://www.nws.noaa.gov/mdl/synop/products.shtml_

- ...and now for Judy’s talk about LAMP

## “Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts” (NRC Report 2006)

-   “By providing mainly single­valued categorical information, the hydrometeorological prediction community denies its users much of the value of the information it produces—information that could impart economic benefits and lead to greater safety and convenience for the nation.”
-  “With the availability of uncertainty information, users— each with their own sensitivity to costs and losses and with varying thresholds for taking protective action—could better decide for themselves whether to take action and the appropriate level of response to hydrometeorological situations.”

## Slide 42

## Localized Aviation MOS Program (LAMP) Background

- LAMP is a system of objective analyses, simple models, regression equations, and related thresholds which together provide guidance for sensible weather forecasts
- LAMP acts as an update to MOS guidance
- Guidance is both probabilistic and deterministic
- LAMP provides guidance for aviation elements
- LAMP bridges the gap between the observations and the MOS forecast
- Good quality recent surface observations help to decrease the uncertainty in the short term.  As the observations become less predictive later in the forecast period, the uncertainty increases.
- Verification shows improvement on MOS in the first hours, then skill comparable to MOS

## Localized Aviation MOS Program (LAMP) Background

- LAMP guidance
- goes out 25 hours in 1 hour projections
- Station Guidance
- All elements
- ~1600 stations
- CONUS, Alaska, Hawaii, Puerto Rico
- Gridded Guidance
- Thunderstorms: Probability/Best Category Y/N of thunderstorm occurrence in a 2 hour period in a 20km gridbox
- CONUS
- Eventually will run 24 times a day (every hour)

## 1-3 hr LAMP Thunderstorm forecast

## 1-3 hr LAMP Thunderstorm forecast

- Verifying lightning

- LAMP Thunderstorm Probability

## 11-13 hr LAMP Thunderstorm forecast

## LAMP Verification

## GFS LAMP Bulletin

**KRDU   GFS LAMP GUIDANCE   9/14/2007  2100 UTC**
**UTC  22 23 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22**
**TMP  75 74 72 72 71 71 70 70 69 68 68 67 67 67 67 69 71 73 74 75 75 76 76 76 75**
**DPT  72 72 72 71 71 71 70 70 69 68 68 67 67 67 67 67 65 64 63 61 59 58 57 55 55                                       **
**WDR  10 11 09 08 08 11 15 25 26 27 28 29 31 30 31 35 36 01 01 36 01 36 36 01 02                                       **
**WSP  04 04 03 04 04 04 03 03 03 04 03 03 02 02 02 04 06 07 08 07 08 08 08 08 07                                       **
**WGS  NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG NG                                       **
**PPO  52 48 61 35 33 29 32 28 20 19 16 14  8  5  4  2  0  0  0  0  1  2  1  1  0                                       **
**PCO   Y  Y  Y  Y  Y  Y  Y  N  N  N  N  N  N  N  N  N  N  N  N  N  N  N  N  N  N                                       **
**P06                          57                27                 6                                                   **
**TP2        33 14  1  0  1     0     0     0     0     0     0     0     0     0                                       **
**TC2         Y  N  N  N  N     N     N     N     N     N     N     N     N     N                                       **
**POZ   0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0                                       **
**POS   0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0                                       **
**TYP   R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R  R                                       **
**CLD  OV OV OV OV OV OV OV OV OV OV OV OV OV OV OV BK SC SC SC SC SC SC SC SC FW                                       **
**CIG   6  6  6  6  6  6  6  5  5  4  4  4  4  4  5  6  8  8  8  8  8  8  8  8  8                                       **
**CCG   6  6  6  5  5  5  5  5  4  4  4  4  4  4  5  5  5  5  5  6  6  6  6  6  6                                       **
**VIS   7  7  7  7  6  7  5  5  5  6  6  6  6  5  6  7  7  7  7  7  7  7  7  7  7                                       **
**CVS   6  5  7  5  5  5  5  5  5  5  5  5  5  5  5  5  7  7  7  7  7  7  7  7  7                                       **
**OBV   N  N  N  N  N  N  N BR BR BR BR HZ BR BR BR  N  N  N  N  N  N  N  N  N  N****                                       **

## Thresholding

-   LAMP makes best category deterministic forecasts based on the probabilities and the thresholds
-   Thresholds are determined based on achieving unit bias or maximizing the threat score within a bias range
-   Thresholds can help interpret the probabilities
-   Thresholds vary by location, projection, cycle time, and season
-   Technique identical to MOS thresholding

## Thresholding

## Thresholding

- Y

- Y

- Y

- Y

- N

- N

- N

## Pieces of the puzzle

## Pieces of the puzzle

## Pieces of the puzzle

## LAMP Probabilities

| Probability of: | Event |
| --- | --- |
| Liquid Equivalent Precip. ≥ 0.01 inch during past 6 hours/12 hours | Yes/No |
| Precipitation occurring on the hour | Yes/No |
| Precipitation type 
(Conditional on Precipitation) | Freezing
Frozen
Liquid |
| Precipitation Characteristics
(Conditional on Precipitation) | Drizzle
Continuous
Showers |

## LAMP Probabilities

| Probability of: | Event |
| --- | --- |
| Thunderstorms during 2 hr period in 20km gridbox | Yes/No |
| Total Sky Cover | 0/8 (Clear)
1/8 – 2/8 (Few)
3/8 – 4/8 (Sct)
5/8 – 7/8 (Bkn)
8/8 (Ovc) |
| Obstruction to Vision | No obstruction to vision
Haze/Smoke
Mist
Fog
Blowing Phenomena |

## LAMP Probabilities

| Probability of: | Event |
| --- | --- |
| Ceiling Height | < 200 feet
200 – 400 feet
500 – 900 feet
1000 – 1900 feet
2000 – 3000 feet
3100 – 6500 feet
6600 – 12,000 feet
> 12,000 feet |
| Ceiling Height (Conditional on Precipitation) | Same as above |

## LAMP Probabilities

| Probability of: | Event |
| --- | --- |
| Visibility  | < 1⁄2 mile
< 1 mile
< 2 miles
< 3 miles
≤ 5 miles
≤ 6 miles |
| Conditional Visibility (Conditional on Precipitation) | Same as above |

## LAMP Probabilistic Products

- SBN/NOAAPort/NWS FTP server products:
- Alphanumeric bulletin guidance
- Station guidance in BUFR format
- Contains all probabilities made by LAMP
- Gridded thunderstorm guidance in GRIB2 format (NDGD)
- 2hr thunderstorm probabilities
- Graphical products on weather.gov:
- Gridded thunderstorm images, including probabilities
- Station plots of POPO
- Meteograms, including probabilities found in bulletin

- LAMP Thunderstorm Probabilities

## LAMP Meteogram Products

## LAMP Probability of PrecipitationAvailable on weather.gov/mdl/lamp

## 827

**827**

**2642**

**4348**

**5189**

**4171**

**3625**

**3684**

**5574**

**14848**

**56727**

**145522**

**165669**

**100%**

## 127

**127**

**350**

**1268**

**2145**

**3175**

**3815**

**3233**

**7822**

**15090**

**48684**

**161115**

**167357**

**100%**

## June 8, 2007

- GMOS 03h forecast
- Available ~16:45 UTC
- Valid 18-21 UTC

## June 8, 2007 1500 UTC LAMP forecast

- LAMP 02h forecast
- Available ~15:45 UTC
- Valid 17-19 UTC

- Verifying Lightning Strikes

## June 8, 2007 1800 UTC LAMP forecast

- LAMP 02h forecast
- Available ~18:45 UTC
- Valid 19-21 UTC

- Verifying Lightning Strikes

## Uses of LAMP probabilities

-   As input to AvnFPS in making Terminal Aerodrome Forecasts (TAF)
-   “PROB: Probability of occurrence of a thunderstorm or other precipitation event, with associated weather elements as necessary (wind, visibility, and/or sky condition) whose occurrences are directly related to, and contemporaneous with, the thunderstorm or precipitation event. Only PROB30 (30% probability of the specified element occurring) groups will be used in NWS TAFs.” (NWSI 10-813)
-   As guidance to WFO forecasters in making the TAFs and to AWC forecasters in making the convective products
-   Aviation Planning – Keith and Leyton (WAF August 2007) found “that utilizing statistically derived probabilistic forecasts to determine fuel carriage results in a significant cost savings compared to the deterministic TAF forecasts.”
-   WFO smart tools (e.g., Charleston WV)

## GFS LAMP Status

- Operational Status:
- First 4 cycles operational July 2006

| 00 | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |

- Available LAMP cycles (UTC)

## GFS LAMP Status

- Operational Status:
- First 4 cycles operational July 2006
- Next 4 cycles operational May 2007

| 00 | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |

- Available LAMP cycles (UTC)

## GFS LAMP Status

- Operational Status:
- First 4 cycles operational July 2006
- Next 4 cycles operational May 2007
- 4 new cycles to be operational September 26, 2007

| 00 | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |

- Available LAMP cycles (UTC)

## GFS LAMP- the Future

- Total sky cover - Station guidance

- Total sky cover - Gridded guidance

## GFS LAMP- the Future

- POPO - Station guidance

- POPO - Gridded guidance

## MDL MOS and LAMP Products on the Web

- MOS Products
- http://www.nws.noaa.gov/mdl/synop/products.shtml
- LAMP Products

- http://www.weather.gov/mdl/lamp/