# Presentation: 369431

## WDTB Winter Weather Workshop July 2003Ensemble Forecasting 

- The quick-reference/main-points version
- Stephen Jascourt	Stephen.Jascourt@noaa.gov
***COMET*** NWP resource
- Some slides prepared by Jun Du (NCEP) and Bill Bua (COMET), and their material included contributions from Steve Tracton (formerly NCEP) and Zoltan Toth (NCEP)

## What kinds of information can an ensemble add to the forecast?

**INFO on DISTRIBUTION of SCENARIOS: **
**- how many scenarios       - how likely is each    - how sharply defined is each**

**Increased accuracy!**

- Forecast skill: #1=humans, #2=ensemble means, #3=individual model runs

- Ensembles identify high vs. low predictability

*** ***_Ensembles vs. one higher-resolution run_** **
**even small timing and placement errors can be significant in attempt to accurately forecast details; orographic local winds and precip highly sensitive to synoptic wind direction (see Mass, et al., 3/02 BAMS****!!!****).**
***    “******One detailed mesoscale model based forecast  could  allow the  user to make highly specific and detailed inaccurate forecasts.” (after Grumm******)***

## GFS

**GFS**
**00, 06, 12, 18 UTC**
- “MRF” with late data cutoff is gone, fields labeled “MRF” are same as fields labeled “AVN”

**Global**
**Ensembles**

**2002 until Oct 29           After Oct 29, 2002**

-           84h       180h             384h
-        3 1⁄2 d      7 1⁄2 d              16 d

-           84h       180h             384h
-        3 1⁄2 d      7 1⁄2 d              16 d

-           84h       180h             384h
-        3 1⁄2 d      7 1⁄2 d              16 d

-           84h       180h             384h
-        3 1⁄2 d      7 1⁄2 d              16 d

- T170
- L42

- T62  L28

- T254
- L64

- T170
- L42

- T126
- L28

- T126
- L28

- T62  L28

- T126
- L28

- T62  L28

- 11 members (1 control, 10 perturbations)    11 members (1 cont., 10 pert)
-             00 UTC, 12 UTC                             00 UTC, 06 UTC, 12 UTC, 18 UTC

** ****until mid-2003                  Starting sometime 2003**

**GFS and GLOBAL ENSEMBLES**

- Note ensembles run at lower resolution and resolution decreases during the forecast period

## Short-Range Ensemble Forecasts (SREF)

- _NOW_
-  5 Eta 48 km (control + 2 perturbation pairs)
-  5 Regional Spectral Model 48 km (control + 2 perturbation pairs)
-    [RSM has very old AVN/MRF physics, not upgraded version]
-  5 Eta members using Kain-Fritsch convective parameterization
-  21, 09 UTC to 63 hours, in time for your use with 00, 12 UTC Eta
- _CHANGES PLANNED FOR FALL/WINTER 2003_
-  Resolution change to 32 km grid-spacing
-  RSM physics upgraded to current GFS physics
-  BUFR sounding and surface data available for all members
-  2 configurations being tested, best will be used
- Same as current
- Many different convective parameterizations
- RUC and ARPS *might maybe* *perhaps* get added to SREF

## Types of Products, How to use

## http://wwwt.emc.ncep.noaa.gov/mmb/SREF/SREF.html

**http://wwwt.emc.ncep.noaa.gov/mmb/SREF/SREF.html**
**NCEP Short range ensembles on the web. ****Note wwwt address may change to www**

## Mean and Spread:  Interpretation

**“****spread”**
**= standard deviation.**
**Highest – lowest is bigger than spread**

**Notes:**
Idealized interpretation of spread displays (dark contours represent ensemble
mean and shaded areas are spread).  Rarely will the picture look this clear!

## Global

_**3 day forecast from 00 UTC 11/2/01**_

**Phase uncertainty - where will the trough axis be?**

**Depth uncertainty - how strong will trough be?**

**SD -**
**meters**

**Global**

## Mean and Spread:  Advantages

**Compact communication **
**Can see field over entire domain**
**Ensemble mean **_***on average***_** has greater skill than any individual member **
**Spread (sample standard deviation) quantifies the degree of uncertainty**

## Assumes normal distribution of forecasts (bell curve with maximum likelihood at mean)
Mean may hide important details
Bi- or multi-modal solutions
Timing problems in prediction of features
COMMON PROBLEM: cyclone/shortwave timing mismatches – all members have wave/cyclone but shows up much weaker in ensemble mean (phase cancellation)
Precipitation forecasts (particularly where convective precipitation is expected to be important – see next slide)
Can use spread as guide to where mean may not be communicating the correct information, and use additional tools to make further assessments

**Assumes normal distribution of forecasts (bell curve with maximum likelihood at mean)**
**Mean may hide important details**
**Bi- or multi-modal solutions**
**Timing problems in prediction of features**
**COMMON PROBLEM: cyclone/shortwave timing mismatches – all members have wave/cyclone but shows up much weaker in ensemble mean (phase cancellation)**
**Precipitation forecasts (particularly where convective precipitation is expected to be important – see next slide)**
**Can use spread as guide to where mean may not be communicating the correct information, and use additional tools to make further assessments**

_***Mean and Spread:  Limitations***_

## Example where ensemble mean is misleading

- <date/time>

- 4 ensemble members: each one has one precip bulls eye of 1.0” but each predicts the bulls eye in a different location

- Ensemble mean has 4 precip bulls eyes, each of 0.25”

## “Spaghetti” Diagram Interpretation

- Phase uncertainty

- Amplitude uncertainty

## “Spaghetti” Diagram Interpretation: Clustering

- Clustering

- Ensemble mean

**Notes:**
In this idealized drawing, there are two clusters of preferred forecasts for the hypothetical 500-hPa height line.  One indicates a strongly amplified trough, while the other indicates flat flow.  Note that the ensemble mean masks the fact that there are two preferred solutions.  Ensemble spread would indicate lots of uncertainty, but nothing more specific than that.

## global

**Uncertain location of **
**incoming western trough**

**From CDC web site:  **
**http://www.cdc.noaa.gov/map/images/ens/ens.html**

- Uncertain amplitude
- of eastern trough

- _3-day forecast from 00 UTC 11/2/01, spaghetti diagram for ensemble_

**global**

## “Spaghetti” Diagrams:  Advantages

**Avoids the assumption of normally distributed data and pitfalls thereof**
**Can tell if ensemble mean, if present, is representative of the ensemble as a whole**
**Shows spread among ensemble members *****and***** whether there is clustering of members around two or more forecasts**
**Shows mode (I.e. most frequently occurring solution)**
**Indicates outliers which may overly influence the ensemble mean and spread**

## “Spaghetti” Diagrams:  Limitations

**Limited to one or only a few contours**
**Cannot see full field of interest over the full domain**
**May not choose the right contour (use ensemble mean/spread to make the best choice)**

## PROBABILITY CHARTS

**PROBABILITY CHARTS**
**          ****percentage of ensemble members with value exceeding threshold**

**Percentage of members with QPF > .25”/24h**

**SREF**

## Probability charts:  Advantages

**Depicts probabilities for exceeding critical value in a compact manner**
**Variable of interest is seen over the full domain**
**Uses actual distribution of data from ensemble members to determine probabilities**

## Probability charts:  Limitations

**Do not get information on *****full *****PDF**
**Only know percentage of ensemble members that exceed the value (sampling problem of limited ensemble size)**
**Need to use several threshold values for complete picture **
**Does not depict maximum value**

## Individual Station Plots: MeteogramsEnsemble mean and all members. Experimental, available by fall 2003linked on http://www.emc.ncep.noaa.gov/mmb/research/meso.products.html

- Individual Station Plots: MeteogramsEnsemble mean and all members. Experimental, available by fall 2003linked on http://www.emc.ncep.noaa.gov/mmb/research/meso.products.html

## Precip-type algorithms“ensemble” of ptype from operational Eta. Available during fall/winter linked on http://www.emc.ncep.noaa.gov/mmb/research/meso.products.html

## SREF

**010519/0000V63 SREFX-CMB; SFC LOWS**

*Sequence provides info on envelope of storm tracks*

**SREF**

- Storm tracks

## Spread appears

- high....

- small....

- But what happens when we scale it by recent spread for same location and forecast length (132 hours in this example)?

## ....but actually

- ....but actually
- relatively low for this location / season!

- ....but is actually larger than usual for this location/season

## Calibrated output

- Bias-corrected probabilities
-  match forecast probabilities to **verification stats**
-  example: if 0.5” precip verifies 30% of occasions when 50% of ensemble members QPF > .5”, then PQPF=50% gives CPQPF=30%

- Probability (PQPF) > 0.5” in 24h period       Calibrated probability (CPQPF)

-            wet bias                                           bias removed = lower probabilities

## Relative Measure of Predictability (RMOP)measure of how likely the ensemble mean is,distinguish predictable vs. unpredictable situationshttp://wwwt.emc.ncep.noaa.gov/gmb/ens/index.html (note: wwwt may become www)

- Based on last 30 days of ensemble performance to take into account regime predictability and general model performance
- Ensemble mean and each ensemble member placed in equally likely climatological bins (bins vary seasonally and geographically to account for typical variability)
- RMOP colors with percentage below color bar show the percentile rank of today’s forecast compared to the last 30 days for number of ensemble members agreeing with their ensemble means (“agreeing with” = in the same bin)
- For example, red (90%) means the ensemble distribution has more members in the same bin as the mean than 90% of the cases in the past 30 days, suggesting this is among the most predictable forecasts in the last month
- RMOP probability numbers (above the color bar)
- Calibrated probability that ensemble mean will verify based on how often the ensemble mean verified when the same number of ensemble members were in the bin containing the ensemble mean during the past 30 days

## Block:  Relatively

**Block:  *****Relatively***
**predictable**

- 90% of the time in the last 30 days, **fewer** ensemble
- members fell into the same bin as the ensemble mean
- (I.e. relatively high predictability as we’ve defined it here), ***BUT*** only verified ***43% of the time!***

## Unpredictable

**Unpredictable**
**heights in strong gradient**

**Ridge/Trough**
**Highly predictable**

## SREF Behavior: Current Configuration

**RSM and Eta tend to group into separate clusters (especially for QPF), mostly because...**
**RSM based on GFS analysis, Eta based on Eta**
**SREF Eta tends to have smaller spread than the RSM**
**Eta vs. Eta with Kain-Fritsch convection**
**Increased ensemble spread**
**Could be result of using KF/mass-flux scheme *****or *****use of 4****th**** order (less damping) diffusion scheme**
**Sharper gradients, including in stability parameters**
**Tendency for larger instability**
**Consistent with later triggering of convective scheme than for BMJ**
**But BMJ often has very high surface CAPE over warm water, apparently due to lack of vertical mixing**

## 2 main forecast tips:

**Ensembles cannot compensate for very large analysis errors** – when analysis errors are big, the verification may not lie within the envelope of ensemble solutions. #1 rule when using NWP is always check for problems in the analysis
- Ensemble mean and overall pattern of distribution of ensemble members is ***far***** more consistent run-to-run** than any one individual model

## 		Additional Ensemble Links

**Note: wwwt.emc.ncep.noaa.gov may change to www.emc.ncep.noaa.gov**
**http://eyewall.met.psu.edu/mos/index.html**** - global ensemble MOS**
**http://wwwt.emc.ncep.noaa.gov/gmb/ens/training/.html – CASES!!!**
**http://wwwt.emc.ncep.noaa.gov/gmb/ens/index.html**** - global ensemble home**
**http://wwwt.emc.ncep.noaa.gov/mmb/SREF/SREF.html - SREF home**
**http://eyewall.met.psu.edu/ensembles2/index.html**** - global anomaly**
**http://eyewall.met.psu.edu/SREF/index.html**** - SREF anomaly**
**http://www.meteo.psu.edu/~gadomski/ewall.html**** - maps**
_**http://www.hpc.ncep.noaa.gov/ensembletraining**_** - Pete Manousos’ web page**
_**http://www.ecmwf.int/newsevents/training/rcourse_notes/GENERAL_CIRCULATION/CHAOS/Chaos.html**_

## COMET

**NEW NWP PCU1 ensembles module by Bill Bua, including cases, ready by fall! **
**Ask about NCEP model concerns as they arise! COMET newsgroups**
**http://www.meted.ucar.edu/nwp/newsgroups/index.htm**
**Winter Weather Refresher powerpoint:**
**http://www.meted.ucar.edu/comm_highered/winterpp.ppt**
**(2002 version)**

## PRECIP TYPE – follow-up to Dan Baumgardt’s talk

- Info on the SNRA variable from the Eta model,
- Comparison with the Baldwin algorithm and example skew-T plots from the model with output provided from both methods is shown in the middle of the “What can you expect from the Eta-12?” Visitview teletraining available online at
**http://www.cira.colostate.edu/ramm/visit/eta12.html**
- There are 4 pages of interest beginning with the one titled “Precip type in model and output”.
- It can be viewed directly on the web or you can download the visitview file and view it in “visitlocal” mode