# Presentation: 669265

## Using Ensembles in a Deterministic Forecast Era

- Bernard N. Meisner
- Scientific Services Division
- NWS Southern Region
- Fort Worth, Texas

**National Weather Service**

- The views expressed herein are those of the author and do not necessarily reflect the position of the National  Weather Service.

## Using Ensembles in a Deterministic Forecast Era

**    ****What Information is Available in Ensemble Guidance?**
**Obviously, the ****ensemble mean, spread and extremes****.**
**But what about the results of a ****cluster analysis?**
**Might the ensemble MOS guidance be ****calibrated**** to provide useful information?**
**What about applying**** pattern recognition?**

## Model Flip-Flops

**National Weather Service**

## Model Flips and FlopsHow to define them?

**Model Flips and Flops***How to define them?*

**Current model run**
**lies outside envelope**
**of previous ensemble.**

**Run-to-run change**
**in MOS max/min**
**temperatures >10****o****F.**

**Run-to-run change**
**in MOS max/min**
**temperatures exceeds**
**monthly Mean Absolute Error.**

## Using Ensembles in a Deterministic Forecast Era

**    ****Identifying Model Flip-flops**
**Model flip flops ****usually result from forecast uncertainty,**** rather than radical changes in the initial data. **
**Operational forecast ****will often resemble one of the ensemble members**** ****from a previous model run**** valid at the same time, and almost always fall within the range of forecasts given by the prior ensemble run. **
**The ****ensemble mean and spread**** is a better measure of fundamental run-to-run model forecast change than consecutive operational forecasts. **

## Verification

- 108 Hr
- Forecast

- 102 Hr
- Forecast

- 96 Hr
- Forecast

## Confidence Graph, Spaghetti ChartsHigh & Low Confidence ExamplesFSU/NWS TLH  Web Sitehttp://moe.met.fsu.edu/confidence/

**Sfc Vorticity**

**Sfc Wind Speed**

**Confidence Graph, Spaghetti Charts**High & Low Confidence Examples*FSU/NWS TLH  Web Site*http://moe.met.fsu.edu/confidence/

- Spread of Current
-  Ensemble Run

- Average Ensemble Spread
- for time of year

- Climatological Spread
- NCEP/NCAR Reanalysis

## Using Ensembles in a Deterministic Forecast Era

**Analysis has indicated that forecast errors are greater when the spread of the ensemble is relatively large.**
**But we don’t yet know how to minimize that error, given any additional information that may be contained within the ensemble guidance.**

## Model Flip Flops

**Data Used:**
**MEXMOS Guidance for DFW and PIT Airports (Have also looked at a few other sites in NWS Southern Region.)**
**Flip (and Flop) Criterion:**
**Run-to-run change in Max/Min temperature guidance    of 10****o****F or greater.**

## DFW Max/Min Temperature10oF Threshold

| 
Month | 
Flips | Number Correct | Flip – Flops | Number Correct |
| --- | --- | --- | --- | --- |
| Oct 2005 | 3 | 3 | - | - |
| Nov | 7 | 3 | - | - |
| Dec | 9 | 4 | 7 | 4 |
| Jan 2006 | 8 | 5 | - | - |
| Feb | 5 | 5 | 1 | - |
| Mar | 10 | 5 | 3 | 2 |
| Apr | 2 | 0 | - | - |
| TOTAL | 42 | 25 (60%) | 11 | 6 (55%) |

## Model Flips...and Flops

**Summary:**
**Flips (and flops) occur during the cool season.**
**Typically occur for *****just one verification time*****.**
**Model flips are rare; flip-flops are very rare.**
**53 Flips; 11 Flops (out of 400+ model runs/7000+ forecasts)**
**Flips are most common for Days 4-6. **
**Flip-flops are most common for Days 5-6.**

## Forecast Opportunities

**National Weather Service**

## Forecast Opportunities

**Data Used:**
**MEXMOS Guidance for DFW and PIT Airports (Have also looked at a few other sites in NWS Southern Region.)**
**Forecast Opportunity Criterion:**
**Max/Min temperature guidance error 10****o****F or greater.**
**Caveat:**
**Smaller errors at certain thresholds can be significant!**
**	****28****o****F vs 33****o****F;   99****o****F vs 104****o****F**

## Forecast OpportunitiesThree Days in August

**National Weather Service**

## August 2006

|  |  | 101
80 | 101
78 | 101
79 | 102
79 | 103
79 |
| --- | --- | --- | --- | --- | --- | --- |
| 103
79 | 94
75 | 100
77 | 103
80 | 104
83 | 105
80 | 103
82 |
| 102
79 | 103
79 | 105
79 | 104
80 | 105
79 | 105
81 | 102
82 |
| 102
82 | 103
83 | 103
80 | 102
77 | 104
81 | 103
83 | 102
82 |
| 96
78 | 83
76 | 89
73 | 92
67 | 97
71 |  | Climo
94.8
74.0 |

## August 2006

|  |  | 101
80 | 101
78 | 101
79 | 102
79 | 103
79 |
| --- | --- | --- | --- | --- | --- | --- |
| 103
79 | 94
75 | 100
77 | 103
80 | 104
83 | 105
80 | 103
82 |
| 102
79 | 103
79 | 105
79 | 104
80 | 105
79 | 105
81 | 102
82 |
| 102
82 | 103
83 | 103
80 | 102
77 | 104
81 | 103
83 | 102
82 |
| 96
78 | 83
76 | 89
73 | 92
67 | 97
71 |  |  |

## August 24, 2006

**Period One MOS Guidance:**
***Operational:*****   94  OVC**
***Cntrl:***** ****	****	****  99   PC**
***P1–P4*****   100 PC****	****98 PC****	****  ****	****  99 PC****	****98 OVC**
***P4-P8*****      99 PC****	****99 PC****	****	****100 PC****	****99 PC**
***P9-P12*****    97 PC****	****98 PC****	****	****  99 PC****	****99 PC**
***P13-P14*****  99 PC****	****99 PC**
**	****While the OVC sky cover was triggered in the operational run, it was only triggered in one ensemble member.**

## August 2006

|  |  | 101
80 | 101
78 | 101
79 | 102
79 | 103
79 |
| --- | --- | --- | --- | --- | --- | --- |
| 103
79 | 94
75 | 100
77 | 103
80 | 104
83 | 105
80 | 103
82 |
| 102
79 | 103
79 | 105
79 | 104
80 | 105
79 | 105
81 | 102
82 |
| 102
82 | 103
83 | 103
80 | 102
77 | 104
81 | 103
83 | 102
82 |
| 96
78 | 83
76 | 89
73 | 92
67 | 97
71 |  |  |

## August 12, 2006

**Period One MOS Guidance:**
***Operational:*****   94  OVC**
***Cntrl:***** ****	****	****  94   OVC**
***P1–P4*****     93 OVC****	****94 OVC  ****	****  94 OVC****	****93 OVC**
***P4-P8*****      96 OVC****	****94 OVC****	****  98 OVC****	****93 OVC**
***P9-P12*****    95 OVC****	****93 OVC****	****  96 OVC****	****93 OVC**
***P13-P14*****  96 OVC****	****95 OVC**
**	****All forecast runs triggered the OVC sky cover.**

## August 2006

|  |  | 101
80 | 101
78 | 101
79 | 102
79 | 103
79 |
| --- | --- | --- | --- | --- | --- | --- |
| 103
79 | 94
75 | 100
77 | 103
80 | 104
83 | 105
80 | 103
82 |
| 102
79 | 103
79 | 105
79 | 104
80 | 105
79 | 105
81 | 102
82 |
| 102
82 | 103
83 | 103
80 | 102
77 | 104
81 | 103
83 | 102
82 |
| 96
78 | 83
76 | 89
73 | 92
67 | 97
71 |  |  |

## August 22, 2006

**Period One MOS Guidance:**
***Operational:*****   94  OVC**
***Cntrl:***** ****	****	****  95   OVC**
***P1–P4*****     95 OVC****	****95 OVC  ****	****  95 OVC****	****95 OVC**
***P4-P8*****      96 PC****	****95 OVC****	****  95 OVC****	****95 PC**
***P9-P12*****    96 PC****	****95 OVC****	****  97 PC****	****94 OVC**
***P13-P14*****  95 OVC****	****97 PC**
**	****All but five of the forecast runs triggered the OVC sky cover.  Runs with PC sky cover still substantially underestimated the maximum temperature of 103****o****F.**

## Forecast Opportunities

**Forecast Opportunities:**
**Typically occur for *****calendar days***** rather than *****model runs*****.**
**Are most common for Days 3-7. **
**Frequently occur for days on which the observed max/min temperature departs substantially from the climatological norm.**
**MEXMOS guidance typically underestimates the observed departure from normal.**
**Rarely occur for days when the temperature change from the previous day is large.**
**GFS seems to handle these events well.**

## Calibrated Ensemble MOS PoP Guidance

**National Weather Service**

## Calibrated PoP Guidance

**WFO Jackson, Miss. Procedure:**
**Initialize each forecast package with 0000 UTC MEX MOS guidance.**
**Only change grids when/where necessary.*****(Forecast Opportunities)***
**Increase forecast PoP when ensemble mean PoP exceeds predetermined thresholds.**
**Decrease forecast PoP when emsemble mean PoP is below predetermined thresholds.**
**Technique applicable for cool season only.**

## Slide 25

## Slide 26

## Slide 27

## Slide 28

## Improvement over MOSKJAN PoP 2005 vs 2004

**Improvement over MOS***KJAN PoP 2005 vs 2004*

**2005**

**2004**

## Calibrated PoP Guidance

**Results:**
**Big improvement over previous year.**
**Removed previous dry bias for all forecast periods.**
**Average forecast PoP was greater than MOS guidance on rainy days for all forecast periods.**
**Average forecast PoP was less than MOS guidance on dry days for all forecast periods.**

## Using Ensembles in a Deterministic Forecast Era

**Even in a probabilistic forecast era, some users will continue to require the most likely scenario.  Will that always be the ensemble mean?**
**As the size of the ensembles increase it will not be possible to manually examine the output from each member. ****(No more spaghetti charts!)**
**How might we glean all the useful information from an ensemble – much more than just the means and extremes?**

## Questions?

- mailto: bernard.meisner@noaa.gov

**National Weather Service**