# Presentation: 071924

## PREDICTING PREDICTABILITY

***Zoltan Toth***
**Environmental Modeling Center**
**NOAA/NWS/NCEP,**
- Ackn.: Yuejian Zhu (1), Richard Wobus (1), Mozheng Wei (2)
- (1) : SAIC at NCEP/EMC, Washington, US (www.emc.ncep.noaa.gov)
- (2) : UCAR Visiting Scientist, NCEP/EMC, Washington, US
*http://wwwt.emc.ncep.noaa.gov/gmb/ens/index.html*

## OUTLINE / SUMMARY

**DEFINITION OF PREDICTABILITY**
- No universally accepted form?
**COMPLEX MEASURE OF PREDICTABILITY**
- What is predictable (Probabilistic forecast format)
- Forecast skill (Resolution)
**PREDICTING PREDICTABILITY**
- Practical aspect (Dynamical-statistical error variance prediction)
- Theoretical aspect (Predictability depends on our ever expanding knowledge)
**HOW PREDICTABILITY CAN BE ENHANCED? **
- Capture flow dependent variations in predictability
- Use “high resolution” forecast in probability space
- Consider details in pdf (Bimodality)
**POSSIBLE FUTURE ENHANCEMENTS**
**CAPTURE MODEL RELATED FLUCTUATIONS IN FORECAST UNCERTAINTY**
- Represent model errors due to
- Structural
- Parametric
- Closure type uncertainties
***NEED (COSTLY AND) COMPREHENSIVE APPROACH?***

## WHAT IS PREDICTABILITY?AND FORECASTING?

- DISCUSSION AT SEPT. 2002 ECMWF WORKSHOP –
***No generally accepted, clear definition?***
- Shukla:
- Predictability – Just talking about things, without really doing it, theory
- Forecasting – The REAL thing, telling what’s going to happen
- Palmer:
- Predictability – Has practical aspect, probabilistic forecasting, link with users
- Webster:
- Predictability – Explore what can be skillfully predicted
- Simple measures of predictability:
- Linear –
- Global or local Lyapunov Vectors (LVs)
- Finite-Time Normal Modes (FTNM, Frederiksen & Wei)
- Singular vectors (SVs)
- Nonlinear -
- Bred vectors (Nonlinear LVs)
- Nonlinear SVs, etc

## WHAT IS PREDICTABILITY?WHAT IS FORECASTING?

**PREDICTABILITY ****- STUDYING WHAT IS PREDICTABLE**
**	*****BASED ON TWO FACTORS*****:**
***INHERENT NATURE OF FLOW***
- 	**Theoretical approach** – Have to make oversimplifying assumptions (see measures)
- 				  Provides general information, limited insight
***KNOWLEDGE / REPRESENTATION OF***
- 				Initial state of system
- 				Laws governing evolution of system
- 	**Practical approach** – 	Tell every day what is	*Predictable? *
*	**	**	**	**	**	**	**Expected error?*
*	**	**	**	**	**	**	**Forecast uncertainty? =>*
- 				PROBABILISTIC FORECASTING
- 			**FORECASTING** (IN ITS FULL SENSE) –
**PROBABILISTIC, WITH CASE SPECIFIC PREDICTABILITY INFORMATION =>**
***ASSESSMENT OF PREDICTABILITY IS PART OF FORECASTING***
*“**NO FORECAST IS COMPLETE UNLESS PROVIDED IN PROBABILISTIC FORMAT”*
- EXTRA INFORMATION FOR USERS?

## “PREDICTING PREDICTABILITY”? 

*Don’t know what organizers had in mind...*
**PRACTICAL INTERPRETATION: **
- Given **current **probability forecast AND distribution of observing locations at **future** time
- 				**Predict how forecast uncertainty will change**
**	****	****	****	**Dynamical-statistical methods
*APPLICATION *– Targeted observations (Bishop et al., Berliner et al.)
**THEORETICAL INTERPRETATION:**
- Predictability is strongly linked with forecasting and depends on our knowledge of:
- 		Initial conditions
- 		Governing equations
- Given **current** level of predictability, and expected advances that lead to **future** observing, data assimilation, and forecast systems –
- 				**Predict how predictability will change in 50 (100) years**
**	****	****	****Can’t do this – Instead:**
**APPROACH:****	****Look at predictability using different existing forecast methods**
**	****	****	****Assess how improvements contribute to enhanced predictability**
**	****	****	****Speculate what advances can be expected**
***PHILOSOPHICAL ASPECT***** – **
- 	**PREDICTABILITY DEPENDS ON OUR UNDERSTANDING OF NATURE**

## HOW TO MEASURE PREDICTABILITY?

**RELIABILITY ****– Lack of systematic error**
**	**(No conditional bias)
***CAN BE statistically corrected***
***       ******(assuming stationary processes)***

***USE FORECAST SKILL MEASURES***
***	****Assume perfect reliability – Skill is measured by resolution*

**RESOLUTION ****– Different fcsts precede**
**    ****different observations**
***CANNOT be statistically corrected -***
***INTRINSIC VALUE OF FCST SYSTEM ***

- For perfectly reliable fcsts, resolution = ensemble spread = spread in observations =>
**Perfect predictability** = only 0 & 100% probabilities used, and always correct
**No predictability** = No matter what we forecast, climate distribution is observed

## Slide 7

## PROBABILISTIC FORECASTING 

**Based on**** SINGLE FORECAST** –
- 	* *
***One integration with an NWP model, *****combined with past verification statistics**

- Does not contain all forecast information
- Not best estimate for future evolution of system
***UNCERTAINTY CAPTURED IN TIME AVERATE SENSE -***
**NO ESTIMATE OF CASE DEPENDENT VARIATIONS IN FCST UNCERTAINTY**

## SCIENTIFIC NEEDS - DESCRIBE FORECAST UNCERTAINTY ARISING DUE TO CHAOS

*Buizza 2002*

## INITIAL CONDITION RELATED ERRORS

***INITIAL CONDITION RELATED ERRORS***
*** ****Sample initial errors*
* **Run ensemble of forecasts*
* **Can flow dependent variations in forecast uncertainty be captured?*
* **May be difficult or impossible to reproduce with statistical methods*

## Slide 11

## Slide 12

## Slide 13

## 144 hr forecast

*144 hr forecast*

*Verification*

- Poorly predictable large scale wave
- Eastern Pacific – Western US

- Highly predictable small scale wave
- Eastern US

## Slide 15

## Slide 16

## Slide 17

## Slide 18

## Slide 19

## Slide 20

## Slide 21

## OUTLINE / SUMMARY

**DEFINITION OF PREDICTABILITY**
- No universally accepted form?
**COMPLEX MEASURE OF PREDICTABILITY**
- What is predictable (Probabilistic forecast format)
- Forecast skill (Resolution)
**PREDICTING PREDICTABILITY**
- Practical aspect (Dynamical-statistical error variance prediction)
- Theoretical aspect (Predictability depends on our ever expanding knowledge)
**HOW PREDICTABILITY CAN BE ENHANCED? **
- Capture flow dependent variations in predictability
- Use “high resolution” forecast in probability space
- Consider details in pdf (Bimodality)
**POSSIBLE FUTURE ENHANCEMENTS**
**CAPTURE MODEL RELATED FLUCTUATIONS IN FORECAST UNCERTAINTY**
- Represent model errors due to
- Structural
- Parametric
- Closure type uncertainties
***NEED (COSTLY AND) COMPREHENSIVE APPROACH?***

## SUMMARY

**PREDICTABILITY (RESOLUTION) IS ENHANCED WHEN **
**Flow dependent fluctuations in uncertainty** captured
- Ensemble mode vs. control forecast
- Stronger effect at longer lead times
**Detailed** (and not bivariate) **probability distribution** is used
- Stronger effect at shorter lead times
- Only broad features of pdf, or details also matter?
**Bi- and multimodality** appears to contribute to ensemble skill
***NCEP ENSEMBLE REPRESENTS ONLY INITIAL VALUE RELATED UNCERTAINTY***
***CAN VARIATIONS IN FORECAST UNCERTAINTY DUE TO MODEL IMPERFECTNESS BE ALSO CAPTURED?***
***WOULD THIS LEAD TO ENHANCED PREDICTABILITY?***
- Lower ensemble mean rms error?
- Increased resolution (use of more close to 0 and 100% fcst probability values)?
- Details in pdf more trustworthy?

## MODEL RELATED FORECAST UNCERTNAINTY 

**MODEL RELATED FORECAST UNCERTNAINTY **
**SOURCES OF UNCERTAINTY - *****MODELS ARE IMPERFECT***:
-  Structural uncertainty (eg, choice of structure of convective scheme)
-  Parametric uncertainty (eg, critical values in parameterization schemes)
-  Closure/truncation errors (temporal/spatial resolution; spatial coverage, etc)
***NOTES****:*
-  Two main sources of forecast errors hard to separate =>
-  Very little information is available on model related errors
-  Tendency to attribute all forecast errors to model problems
**REPRESENTING MODEL RELATED FORECAST UNCERTAINTY -**
***	******	******	******	******NO THEORETICALLY SATISFYING APPROACH***
** **Change structure of model (eg, use different convective schemes, etc, MSC)
-  Add stochastic noise (eg, perturb diabatic forcing, ECMWF)
-  Works? Advantages of various approaches need to be carefully assessed
-  Are flow dependent variations in uncertainty captured?
-  Can statistical post-processing replicate use of various methods?
-  Need for a
-  more comprehensive and
-  theoretically appealing approach	      _***LOTS OF WORK, & POTENTIAL?***_