# Presentation: 440652

## Huug van den Dool and Suranjana Saha  Prediction Skill and Predictability in CFS(and also some diagnostics)CFS Appreciation Day, Nov, 16, 2004

## Definitions Prediction Skill and Predictability

## Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Problems: a) Sample size! , b) Wait a long time(and funding agents are impatient)

## Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Definition 2: Evaluation of skill of hindcasts; hard, not impossible.Problems: a) Sample size, b) ‘honesty’ of hindcasts

## Example of ‘skill’

## Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Definition 2: Evaluation of skill of hindcasts; hard, not impossible.Definition 3: Predictability of the 1st kind (~ sensitivity due to uncertainty in initial conditions)

## Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Sample size!Definition 2: Evaluation of skill of hindcasts; hard, not impossibleDefinition 3: Predictability of the 1st kind (~ sensitivity due to uncertainty in initial conditions)Definition 4: Predictability of the 2nd kind due to variations in external boundary conditions (AMIP; Potential Predictability; Reproducibility; Madden’s approach)

## Predictability (theoretical/intrinsic) is a ceiling for actual prediction skill. In CFS predictability of 1st and 2nd kind merge.

## CFS forecast:X (space, lead, member ,year)

- Space is 2.5oX2.5o (Z500) or 1oX2o (SST/mask), or 1.875 by Gaussian (Soilw, T2m, Precip)
- Basic data used is monthly mean
- Lead = 0, 8 in units of months; member = 1, 15
- Year = 1981 – 2003 (increases annually)
- Example: ‘Initial’ Month is August (= lead 0);
- Note IC is Jul 11/21/Aug 1 for SST, and Jul 09-13/ 19-23 / Jul 30-Aug3 for atmosphere and soil.
- ‘Member’ 16 is ensemble average
- ‘Member’ 17 is matching observed field
- X =  ( Z500, SST, Soilw, T2m, (Precip)**1/2)

## ASPECTS

- Prediction skill (member i vs member 17)
- Predictability (member i vs member j)
- Monthly mean
- Seasonal mean
- Ensemble average
- Predictability of 1st kind only.

## Two types of climatology plus complications

- Xclim_mdl (space, lead) is average over years and (14 or 15) members, depending.
- Xclim_verif (space, lead) is ave over (same) years for either member 17, or member i, i=15.
- Anomaly = X minus Xclim, whichever is relevant
- Systematic error (SE) is automatically corrected by the above
- CV of the SE correction (exclude from Xclim the member and the year to be verified). Not trivial.

## Diagnostics

- Standard deviation (of monthly mean; individual)
- Degrees of freedom (dof)

## Conclusions (monthly data)

- CFS data is a goldmine.
- CFS has enough (?) data for forecast evaluation (and diagnostics)
- Member i vs member j unifies predictability of 1st and 2nd kind in CFS output
- CFS has some prediction skill. In order of skill: SST, {tropical variables}, soilw, Z500,T2m, Precip
- CFS has more predictability (as defined) than pred.skill, but ceiling is ‘low’ in mid-latitudes.
- Seasonality (no surprise)

## To do:

- Identify interdecadal skill source (if any)
- Identify soil moisture skill source (are models still too strong on local effects? How about non-local effects)
- Daily data for the finer temporal scales in skill/predictability.
- Why do models like CFS have predictability in so few d.o.f. (and is that really all there is)
- Further ideas about ‘new’ predictability notions