# Presentation: 440651

## Huug van den Dool and Suranjana Saha  Prediction Skill and Predictability in CFS

## Definitions Prediction Skill and PredictabilityOpinion: Literature fuzzies up ‘predictability’ vs ‘prediction skill’

## Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Problems: a) Sample size! , b) Wait a long time(and funding agents are impatient)

## Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Definition 2: Evaluation of skill of hindcasts; hard, not impossible.Problems: a) Sample size, b) ‘honesty’ of hindcasts

## Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Definition 2: Evaluation of skill of hindcasts; hard, not impossible.Definition 3: Predictability of the 1st kind (~ sensitivity due to uncertainty in initial conditions)

## Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Sample size!Definition 2: Evaluation of skill of hindcasts; hard, not impossibleDefinition 3: Predictability of the 1st kind (~ sensitivity due to uncertainty in initial conditions)Definition 4: Predictability of the 2nd kind due to variations in external boundary conditions (AMIP; Potential Predictability; Reproducibility; Madden’s approach)

## Predictability (theoretical/intrinsic) is a ceiling for actual prediction skill.Any other ‘kinds’ of predictability?

## CFS forecast:X (space, lead, member ,year)

- Space is 2.5oX2.5o (Z500) or 1oX2o (SST/mask), or 1.875 by Gaussian (Soilw, T2m, Precip)
- Basic data used is monthly mean
- Lead = 0, 8 in units of months; member = 1, 15
- Year = 1981 – 2003 (increases annually)
- Example: ‘Initial’ Month is August (= lead 0);
- Note IC is Jul 11/21/Aug 1 for SST, and Jul 09-13/ 19-23 / Jul 30-Aug3 for atmosphere and soil.
- ‘Member’ 16 is ensemble average
- ‘Member’ 17 is matching observed field
- X =  ( Z500, SST, Soilw, T2m, Precip)

## ASPECTS

- Prediction skill (member i vs member 17)
- Predictability (member i vs member j)
- Monthly mean
- Seasonal mean
- Ensemble average
- Predictability of 1st kind only.

## Two types of climatology plus complications

- Xclim_mdl (space, lead) is average over years and (14 or 15) members, depending.
- Xclim_verif (space, lead) is ave over (same) years for either member 17, or member i, i=15.
- Anomaly = X minus Xclim, whichever is relevant
- Systematic error (SE) is automatically corrected by the above
- CV of the SE correction (exclude from Xclim the member and the year to be verified). Not trivial.

## Prediction Skill Monthly

## Conclusions (monthly data)

- CFS data is a goldmine.
- CFS has enough (?) data for forecast evaluation (and diagnostics)
- Member i vs member j unifies predictability of 1st and 2nd kind in CFS output
- CFS has some prediction skill. In order of skill: SST, {tropical variables}, soilw,T2m, Precip
- CFS has some more predictability (as defined), but ceiling is ‘low’ in mid-latitudes.
- Seasonality (no surprise)

## To do:

- Identify interdecadal skill source (if any)
- Identify soil moisture skill source (are models still too strong on local effects? How about non-local effects)
- Daily data for the finer temporal scales in skill/predictability.
- Why do models like CFS have predictability in so few d.o.f. (and is that really all there is)
- Further ideas about ‘new’ predictability notions

## A case for the importance of knowing the effective number of degrees of freedom (edof) in which we have forecast skill.Considerations:-) physical models have one clear strength: they can execute the non-linear terms-) a model needs at least 3 degrees of freedom to be non-linear (Lorenz, 1960)-) a non-linear model with nominally a zillion degrees of freedom, but skill in only <= 3 dof is functionally linear in terms of the skill of its forecasts - and, to its detriment,  the non-linear terms add random numbers to the tendencies of the modes with predictability. ==> Therefore: Physical models need to have skill in, effectively, > 3 dof before they can be expected to take advantage of non-linearity.  (In a forecast setting). (    Note: not any 3 degrees of freedom will do.)

- ‘Lingering memory’
- Cai+Van den Dool(2005); Schemm et al calibration data set,
- (CFS daily data set will be used also).