# Presentation: 075936

## Multi-Model Ensembles

_**24**__**th**__** March, 2005**_
_**CST Conference Call**_

## MM Ensembles

_**Agenda**_
***Present a short review of some recent papers (Arun Kumar)***
***Multi-model Ensemble Methods for U.S. Seasonal Climate Prediction: CDC Approach (Marty Hoerling)***
***Multi-Model Ensembling: A Comparison of Method (Lisa Goddard)***
***Ensemble Coupled Predictions at COLA (Ben Kirtman)***

## MM Ensembles

_**General Scientific Questions:**_
***Why not use the single-best model?***
***How can a poor model add skill?***
***How much improvement?***
**Show some results from a paper by Hagedorn, Doblas-Reyes and Palmer Part I & II (to appear in a special issue of Tellus devoted to DEMETER-  ***a collection of 7 coupled models run 4 times a year from 1981-2000***)**

## MM Ensembles

_**% Improvement Due to MM ensembles (equal weights)**_
**Variable : SST (over Nino 3.4)**
**Solid Bar – 1 Month lead**
**Open Bar – 3 Month lead**
**Period – (1980:2000)**
_***The best model is : season and lead dependent***_

## MM Ensembles

_**% Improvement Due to MM ensembles (equal weights)**_
**Solid Bar – SST**
**Open Bar – MSLP**
**Forecast – May ICs, One month lead**
**Period – (1980:2000)**
_***The best model is : variable and region dependent***_

## MM Ensembles

_**% Improvement Due to MM ensembles (equal weights)**_
**Solid Bar – SST (tropics)**
**Open Bar – MSLP (tropics)**
**Forecast – May ICs, One month lead, **
**Period – (1980:2000)**
_***The best model is : skill measure dependent***_

## MM Ensembles

_**Deciding which model is “the best” has too much dependency on variable, lead-time, region, skill metric etc.  On the other hand MM is consistently better!!**_
_**Also improves reliability!!**_

## MM Ensembles

_**Methods for MM Ensembles**_
***Simple MM ensembles (equal weights to different AGCMs)***
***Optimal MM ensembles (weights based on past performance of AGCMs)***
***Agreement (between different AGCMs)?***

**Notes:**
Hidden Costs!!

## MM Ensembles

_**Scientific & Practical Issues**_
_***[Are Tier-2 approaches for MM ensemble useful?](slide11.xml)***_
_***[How much improvement due to optimal MM schemes?  Stability of weights? ](slide12.xml)***_
***Other MM ensembles (ECMWF)***
***Perfection ; Pragmatism ; Gain ; ROI***

## MM Ensembles

_**FY07-FY11 CTB Milestones**_
***2Q FY07: Transition GFDL model to CTB***
***Complete experimental testing of tier-1 multi-model ensemble forecast system (EMC+GFDL)***
***4Q FY08: Transition NASA model to CTB***
***2Q FY09: Transition CCSM (NCAR) model to CTB***
***4Q FY10: Complete experimental testing of tier-1 multi model ensemble forecast system (EMC+GFDL+CCSM+NASA)***

**Notes:**
ambitious

## MM Ensembles

## MM Ensembles

**(from**_** **_**Hagedorn et al.) Results indicate that the overall forecast quality of probabilistic predictions (of OMM) is not significantly better improved w.r.t. to SMM due to the lack of robustness of coefficients... Similar conclusions were were found to hold by Kharin and Zwiers (2002) in case of SI predictions, in contrast with the result described in Krishnamurti et al. (1999; 2000)**
**The results described in this paper have two implications that are relevant in an operational environment.  First, calibration of  high quality single-model ensembles and their combination require long hindcast data sets, which are difficult to create...  Second, a wide range of results presented here implies that the areas and variables where calibration and/or combination are likely to introduce some skill gain should be carefully selected.  **