# Presentation: 083301

## Ensemble forecast and its applications

**Yuejian Zhu**
- Environmental Modeling Center
- NCEP/NWS/NOAA
**Shanghai, China**
**October 6th 2006**
- _[http://](http://wwwt.emc.ncep.noaa.gov/gmb/ens/)__[wwwt.emc.ncep.noaa.gov/gmb/ens](http://wwwt.emc.ncep.noaa.gov/gmb/ens/)__[/](http://wwwt.emc.ncep.noaa.gov/gmb/ens/)_
_**http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/**_
**Acknowledgements:**
**Z. Toth and S. Lord (NCEP)**
**R. Buizza (ECMWF) and P. L. Houtekamer(MSC)**

## Contents  

- Introduction and useful references
- Why do we need ensemble forecast
- Methodologies of ensemble model forecast
- Review of statistical ensemble forecast
- The skill of ensemble model forecast
- Examples of high predictable system
- The application of ensemble forecast
- International research projects
- Discussion and conclusions

## Introduction

- Background and ensemble concept
- A few decades ago
- Early application by lag ensemble
- Due to lack of computation resource (long-term)
- Required by extended range forecast
- Especially for week-2 forecast
- Improved data analysis and model physics
- Less room to improve the skills?
- Improved observation system
- More understand of forecast errors

## References

- Zhu, 2005: *“Ensemble forecast: A new approach to uncertainty and predictability”* AAS
- Toth, Talagrand, and Zhu, 2006: *“**The attributes of forecast system”* book chapter. Cambridge University Press
- Toth, Talagrand, Candille and Zhu, 2003: *"Probability and ensemble forecasts"* book chapter.
- Zhu, 2004:*“Probabilistic forecasts and evaluations based on a global ensemble prediction system”** *In book of Observation, theory and modeling of atmospheric variability,
- Zhu, Iyenger, Toth, Tracton and Marchok, 1996:*"Objective evaluation of the NCEP global ensemble forecasting system"** *AMS conference proceeding.
- Toth, Zhu and Marchok, 2001:*"The use of ensembles to identify forecasts with small and large uncertainty".* Weather and Forecasting
- Zhu, Toth, Wobus, Rechardson and Mylne, 2002:*“The economic value of ensemble-based weather forecasts”* BAMS
- Buizza, Houtekamer, Toth, Pellerin, Wei and Zhu, 2005: *“Assessment of the status of global ensemble prediction” *MWR
-  + more related articles

## Deterministic or Probabilistic ?

- Any prediction for the future is uncertainty
- Is _fore-cast_!
- No deterministic answer!
- Must be probabilistic!

## Clear sky, no precipitation

## 2mm/24hrs (30%)

- 2mm/24hrs (30%)

- 20mm/24hrs
- (0%)

- Precipitation

## Slide 8

## Current Hurricane Forecast

## Slide 11

## Slide 12

## Example of cost-loss analysis (economic values)

- Wind sheer damages the airplane
- Un-protect airplane (loss)
- 2-million dollars for each airplane
- Protect airplane (cost)
- 20,000 dollars for each airplane
- For decision makers !!!
- 1:100 cost-loss ratio for this case
- Probabilistic forecast and forecast reliability
- Typhoon Mai-sha affected Beijing City
- Un-protect (may loss)
- Flooding, traffic and others.
- Protect (definitely cost)
- Activities will be cancelled
- Labors cost
- Others
- Scientific decision ???
- Anyone counts this ratio?

## Decision Theory Example

- Critical Event: sfc winds > 50kt
- Cost (of protecting):   $150K
- Loss (if damage ):  $1M

*Hit*
*False*
*Alarm*

***Miss***
*Correct*
*Rejection*

**YES          NO**

**YES **
**NO**

- _Forecast?_

- _Observed?_

_**Decision Theory Example**_

- $150K

**$1000K**

- $150K

- $0K

## Prob. Evaluation (cost-loss analysis)

- Based on hit rate (HR) and false alarm (FA) analysis
- .. Economic Value (EV) of forecasts

_*Ensemble forecast*_

_*Deterministic forecast*_

_*Average 2-day advantage*_

## Prob. Evaluation (useful tools)

- ... Small and large uncertainty.
-     _***1 day (large uncertainty) ***_= _***4 days (control)***_ =_*** 10-13 days (small uncertainty)***_

## Slide 17

## Slide 18

## Slide 19

## The skill of global ensemble

- Simple measurements
- Ensemble mean ( like deterministic forecast)
- PAC, RMS and spread
- Distributions
- Talagrand distribution
- Outliers
- Probabilistic evaluations
- Brier skill scores (BSS)
- In terms of resolution and reliability
- Ranked probability skill scores (RPSS)
- Relative operational characteristics (ROC)
- In terms of hitting rate and false alarm rate
- Economic values (EV)

## GEFS configurations

|  | Current | New |
| --- | --- | --- |
| Model | GFS | GFS |
| Initial uncertainty | BV | ETBV |
| Model uncertainty | None | None |
| Tropical storm | Relocation | same |
| Daily frequency | 00,06,12 and 18UTC | same |
| Hi-re control 
(GFS) | T382L64 (d0-d7.5)
T190L64 (d7.5-d16) | same |
| Low-re control (ensemble control) | T126L28 (d0-d16)
00UTC only | T126L28 (d0-d16)
00,06,12 and 18UTC |
| Perturbed members | 10 for each cycle | 14 (20) for each cycle |
| Forecast length | 16 days (384 hours) | same |
| Implementation  | August 17th 2005 | May 30th 2006 |

_**GEFS configurations**_

## Pattern Anomaly Correlation

- Root Mean Square

- Simple Measurement
- For Ensemble mean

- Northern Hemisphere
- 500hPa geopotential height

## Due to model imperfection

- One day advantage

## Outlier – from Talagrand distribution

- Negative
- Spread is too big

- Spread is too small/bias

## Slide 25

## ROC area

- _ROC area_

- May-July 2002

## Brier Skill Scores and decomposition

- _Brier Skill Scores and decomposition_

- Resolution

- Reliability

## NCEP ensemble mean performance for past 5-year

## NCEP ensemble probabilistic performance for past 5-year

- Ranked probabilistic skill scores

- Economic values for 1:10 cost/loss ratio

## High predictable heavy precipitation event

**High predictable heavy precipitation event**

- February 12-13 1997
- (Southern Louisiana flooding)

- Location
- and
- intensity

- GFS    ENS

- GFS made a very good forecast,
- But
- Ensemble made a excellent forecast.

## HIGHLY PREDICTABLE HEAVY PRECIPITATION EVENT (20010313) 

**HIGHLY PREDICTABLE HEAVY PRECIPITATION EVENT (20010313)**

## High Predictable Heavy Precipitation Events (20010113)

**High Predictable Heavy Precipitation Events (20010113)**

## RMOP

**RMOP**

## DETERMINSTIC/PROBABILISTIC  FORECASTQPF .vs. PQPF

- Northern California State Christmas-New Year flooding.
- Winter storm last more than 10 days.
- Total precipitation amount exceeding 660mm over the huge area.
- The homes of 100,000 residents who has been evacuated.
- Some stranded residents has to be rescued by helicopter.
- Caused a lot of damages include road, bridge and resident houses.

- Photo from Washington Post

## 24 hours observation

- 24 hours observation

- GFS             ENS

## Ensemble based Probabilistic Products

- Spaghatti diagram for uncertainty
- Standard/normalized spread, mean
- Relative measure of predictability (RMOP)
- Probabilistic quantitative precipitation forecast (PQPF)
- Probabilistic precipitation types
- Calibrated PQPF
- Hurricane tracks/strike probability
- Anomaly forecasts
- User specified ......
- Calibration !!!

## Ensemble Forecast for Uncertainty (1)

- By Bill Bua

## Ensemble Forecast for Uncertainty (2)

- By Bill Bua

## Ensemble Forecast for Uncertainty (3)

- By Bill Bua

## Ensemble Forecast for Uncertainty (4)

##            1. By using equal climatological bins (e.g. 10 bins, each grid points)2. Counts of ensemble members agree with ensemble mean, (same bin)3. Construct n+1 probabilities for n ensemble members from (2).3. Regional (NH, weighted) Normalized Accumulated Probabilities (n+1)4. Calculate RMOP based on (3), but 30-d decaying average.5. Verification information (blue numbers):  historical average (reliability)

## 10 ensemble members could construct 11 probabilities categories, such as

**10 ****Climatological equally likely bins**

**Ensemble mean**

**10 ****ensemble forecasts**

**The value of ensemble members agree to ensemble mean is 4/10 or 40% (probability)**
**There are 10512 points ( values ) for global at 2.5 * 2.5 degree resolution**

- Example of 1 grid point

- 10 ensemble members could construct 11 probabilities categories, such as
- 0/10 (0%), 1/10(10%), 2/10(20%), 3/10(30%), 4/10(40%), 5/10(50%), 6/10(60%), 7/10(70%), 8/10(80%), 9/10(90%), 10/10(100%)
*Sum of each grid point for above 11 probabilistic categories by area weighted and normalized for global or specified region*
- Get:  0/10     1/10     2/10     3/10     4/10     5/10     6/10     7/10     8/10     9/10     10/10
-          .029     .047     .077     .085     .100     .135     .116     .089     .081     .070       .177           sum of these = 1.0 (1.007 here)
-         2.9%   7.6%   15.3%   23.8%  33.8%  47.3%  58.9%  67.8%  75.9%  82.9%  100%         accumulated values
**There is 30-day decaying average of above values ( ****last line**** ) in the data-base and updated everyday.**
**Assume these are 30-day decaying average values**
**In this case, point value is 4/10, *****RMOP***** value of this point is ****33.8%**

## China

## General public for past 8 years 

- General public for past 8 years

## Specific request

## Slide 46

## Slide 47

## Slide 48

## Ensemble Based Hurricane Track Plots 

- Ensemble Based Hurricane Track Plots

- Frances (08/28)

- Karl (09/18)

## Example of probabilistic forecast in terms of climatology

## ENSEMBLE 10-, 50- (MEDIAN) & 90-PERCENTILE FORECAST VALUES (BLACK CONTOURS) AND CORRESPONDING CLIMATE PERCENTILES (SHADES OF COLOR)

##          The pre-NWP forecast accuracy

**         ****The pre-NWP forecast accuracy**

-  	A schematic illustration of the increase of RMSE with forecast time.  The pre-NWP forecaster started from a persistence forecast which he skillfully extrapolated into the future, converging towards climate for longer ranges

**     ****The time unit can be anything from hours to days depending on the parameter (hours for clouds, days for temperature)**

**persistence**

**meteorologist**

**A********2**

**A**

##    NWP more accurate - but also less

**   ****NWP more accurate - but also less**

**	**A good NWP model is able to simulate all atmospheric scales throughout the forecast. It has the same variance as the observations and the persistence forecasts, which yields an error saturation level 41% above the climate

**persistence**

**meteorologist**

**world’s best NWP**

**A********2**

**A**

## The art of good forecasting

**The art of good forecasting**

-    The way out of the dilemma:
-     	Combine the high accuracy of NWP in the short range with a filtering of the non-predictable scales for longer ranges
- 	This can be done both with and without the EPS

**persistence**

**world’s best NWP**

**meteorologist**

**modified NWP forecast**

**A**

**A********2**

## Thank You!!!