# Presentation: 318484

## Diagnosis of Skill Variability of CPC Long-Lead Seasonal Forecasts: Implications for Production and Use

**Bob Livezey and Marina Timofeyeva**
**NOAA/NWS/OCWWS/Climate Services Division**

**Climate Diagnostics and Prediction Workshop**
**October 21, 2004**
**Madison WI**

## Outline

**Outline**

**Introduction**
**Skill Stratifications**
**Results**
**Conclusions and Lessons**

## Introduction

**Introduction**

**Users should only care about the performance of forecasts that can potentially benefit their decision process**
**Livezey (1990):  There are non-random subsets of seasonal forecasts that were skillful enough to be useful**
** **

## Introduction (Cont.)

**Introduction (Cont.)**

**This talk will make the point :**
***That was made by Livezey (1990)***
***That there are many non-random subsets of forecasts that do not have useful skill ***
***That it is critical for this information to be shared with potential users***
***That skill analyses with different stratifications are highly informative***
***That seasonal forecast production should be dominantly objective***
***That long-lead seasonal forecasts of non-trend signal should only be done on appropriate opportunities, not routinely***

## Displays and Stratifications

**Displays and Stratifications**

**CPC Seasonal Forecasts**
***For 3-equally probable temperature and precipitation classes at 102 Climate Divisions***
***Made every month from 1995 to present for 0.5-, 1.5-, ..., 12.5 month leads***
**Skill Measure:  Modified Heidke Skill Score of Categorized Forecasts**
***1/3 EC’s scored as ‘hits’***
**Displays and Stratifications**
***Summed over all forecasts for each lead for three overlapping seasons at a time (DJF to FMA, FMA to AMJ, etc.) ***
***Stratified further by strong ENSO years vs. other years***
***Mapped for appropriate combinations of leads***

## Temperature

## Temperature

## Temperature

## Results

**Results**

**Seasonal Temperature Die-Aways:**
***Moderate to high national-scale skill confined to Fall/Winter strong ENSO years at short to medium leads***
***Otherwise skill is dominantly modest and level with lead (derived from biased climatologies, ie long-term trend)***
***Worst forecasts are for***
**Fall/Winter at short to medium leads for non strong-ENSO years:  Forecasters subtracting value**
**Summer/Fall at medium to long leads for strong ENSOs:  No remedy except science advances**
***Short-lead forecasts are better now than for the 1960s-80s, ~14 vs ~8 overall, ~20 vs ~13 for the winter***

## Temperature

## Temperature

## Temperature

## Temperature

## Results

**Results**

**Seasonal Temperature Maps:**
***Without a strong ENSO, justification for OND-DJF forecasts is questionable***
***For certain regions/seasons/situations skill is unambiguously useful even for undisciplined, occasional users***

## Precipitation

## Precipitation

## Results

**Results**

**Seasonal Precipitation Die-Aways:**
***Barely useable national-scale skill entirely confined to Fall/Winter strong ENSO years in short to medium leads***
***Otherwise skill is statistically indistinguishable from zero***
***Short-lead forecasts overall seem to be no better now than for the 1960s-80s (but at least are now made two-weeks earlier)***

## Precipitation

## Precipitation

## Precipitation

## Precipitation

## Results

**Results**

**Seasonal Precipitation Maps:**
***Justification for long-lead forecasts is questionable for at least three of six seasonal groups examined***
***For certain regions/seasons/situations skill is unambiguously useful even for undisciplined, occasional users***

## Conclusions and Recommendations

**Conclusions and Recommendations**

**There are non-random subsets of seasonal forecasts that are skillful enough even for undisciplined, occasional users**
**These skills exclusively reflect ENSO and trend signals**
**There are many non-random subsets of forecasts that do not have useful skill **
**We must share this information with potential users**
**Skill analyses with different stratifications are highly informative**
**Seasonal forecast production should dominantly rely on objective exploitation of ENSO and trend with subjective modification only after rigorous criteria are met**
**Routine production of certain long-lead forecasts should be terminated (can be accommodated by transition to separate trend and high frequency forecasts with latter issued on forecast of opportunity basis only)**