# Presentation: 174942

- Improvements in Skill of CPC OutlooksEd O’Lenic and Ken Pelman, NOAA-NWS-Climate Prediction Center 33rd Climate Diagnostics and Prediction Workshop, October 21-24, 2008, Lincoln, Nebraska

**Notes:**
This paper represents an on-going effort at CPC to improve forecasts using an objective consolidation technique developed by Unger, et al, 2008, and was discussed in a less complete form in a paper by O’Lenic, Unger, Halpert and Pelman in J. Weather and Forecasting, June, 2008.

## Introduction

- This paper discusses recent improvements in the skill and coverage of CPC T, P Outlooks.
- The heidke skill score and the percentage of non-EC probabilities are the performance measures.
- s = ((c-e)/(t-e))*100 , -50 < s < 100
- s is the percent improvement over random forecasts

**Notes:**
The goal of the paper is to describe improvements in the skill of 1⁄2-month lead, 3-month outlooks of U.S. Temperature and precipitation, using two very simple skill metrics.  The % improvement that the forecast makes over climatology (HSS), and the % of non-EC forecasts.

- CPC 3-Month Outlook map _lines_ show the probability that the indicated category, B, N, or A,  will occur.
- In blank regions, the probabilities of B, N, A are equal at 1/3 each (EC), and give no forecast.
- Lines show _Non-EC (potentially useful) forecast regions_.
- On a line, probabilities of B and A vary simultaneously and inversely above and below 33.33%, while that of N usually stays at 33.33%.
- The 3 sum to 100% at every point on the map.

**Notes:**
First, I need to provide a bit more context on the form of the forecasts.  In particular the meaning of the maps.

## How CPC Outlooks are Made

- CPC 3-month outlooks are currently made using a combination of at least 5 tools, in consultation with partners.
- From 1995- 2004 these tools were weighted subjectively.  In 2006, an objective consolidation (CON) was introduced, which weights the tools by skill history and spread (Unger et al, 2008).
- Retrospective verification of CON forecasts shows them to be much more skillful than official (OFF) 1995-2004 outlooks (O’Lenic et al, 2008), in both categorical U.S. average skill, and in _coverage by non-equal-chances (non-EC) forecasts_, properties users want.

**Notes:**
More context.  The paper discusses the impact of objectively combining the OCN (Trend), CCA, SMLR and CFS model.  Results indicate an improvement in both skill score and use of non-EC forecast probabilities.

**NEW OTLK**

**Notes:**
This is the actual display of the forecast tools used by the forecaster.  This is publicly available on CPC’s web page.  Tools which are made available with at least a 25 year record of skill can be used in the CON.  The CFS has been used since 2006.

**NEW OTLK**

**Notes:**
The four tools used in this paper, and the consolidation result are shown a the top of the page.

**1⁄2 MO LEAD PRECIPITATION RESULTS**
- Heidke Skill Score (HSS, lines) and Percent Non-EC (colors), Map average % Non-EC.
- A.  OFFICIAL FORECAST (OFF)
- B.  CONSOLIDATION (CON)
- C. DIFFERENCE, CON-minus-OFF, US
-      average% Non-EC
- CON raises US annual average HSS from   9 to 12 compared with OFF

**A**

**B**

**C**

- Area non-ec=53%

**+20%**

- Area non-ec=35%

- Area non-ec=27%

**+8%**

- Area non-ec=14%

- Area non-ec=32%

**+18%**

- Area non-ec=20%

- Area non-ec=36%

**+16%**

- Map color legend, %

- HSS

- HSS

- DIF

- OFF

- CON

- Area non-ec=33%

**Notes:**
The two left-hand sets of maps show two fields.  Contours are the HSS of the OFFicial forecast (TOP), and the CON forecast (BOTTOM).  The shading on these two sets of maps is the percent of the time non-EC forecasts were made from 1995 through 2004.  The maps at lower right is the arithmetic difference between the shaded % non-EC fields, CON-minus-OFF.

## 1⁄2 MO LEAD TEMPERATURE RESULTS

**A**

**B**

**C**

**1⁄2 MO LEAD TEMPERATURE RESULTS**
- Heidke Skill Score (HSS, lines) and Percent
- Non-EC (colors), Map average % Non-EC.
- A.  OFFICIAL FORECAST (OFF)
- B.  CONSOLIDATION (CON)
- C.  DIFFERENCE CON – OFF US
-       average% Non-EC
- CON raises US annual average HSS from
- 22 to 26 compared with OFF

- Area non-ec=41

- Area non-ec=96%

- Area non-ec=46

- Area non-ec=57

- Area non-ec=47

- Area non-ec=78

- Area non-ec=27

- Area non-ec=67

**+11%**

**+31%**

**+40%**

**+55%**

- Map color legend, %

- HSS

- HSS

- DIF

- OFF

- CON

**Notes:**
Same as prior figure, but for temperature.

## GPRA Score Official Skill Metric:48-Mo. Running Mean U.S. Average T HSS

## SUMMARY

- - Outlook prepared subjectively 1995-2004
- - Objective consolidation begun 2006
- - Retrospective verification shows significant increase in CON skill over OFF
- - Western and Eastern P forecasts better than many areas
- - Forecasts are better,  more objective
-    -  Higher categorical skill
-    -  Far fewer “EC” forecasts
- - P HSS rose** **from 9 (OFF) to 12 (CON)   (US ann. mean)
- - T HSS rose from 22 (OFF) to 26 (CON) (US ann. mean)
- - % Non-EC rises in all seasons, >30% for P, >50% for T
- - Official T skill rose starting in  2006 due to use of CON.

## Forecast Evaluation Tool:  Example of a Means to Address Gaps

- What FET and CLIDDSS provide:
- User-centric forecast evaluation and data access and display capability.
- Leveraging of community software development  capabilities.
- Opportunity to DISCOVER, collect, and invest in user requirements.

## FUTURE:  Implement FET at CPC

## FUTURE:  Implement FET at CPC

## FET:A Wide Variety of Skill Renderings

- FET:A Wide Variety of Skill Renderings

- P

- T

- A

- A

- A

- A

- A

- A

- B

- B

- B

- B

- B

- B

## FUTURE of the FET 

- Next 6 months:
- Finalize and implement FET project plan at CPC.
- Ellen Lay (CLIMAS) to train CPC personnel on FET version control and bug tracking at CPC, November 12-14, 2008.
- Necessary software (APACHE TOMCAT, JAVA, Desktop View) acquired and installed at CPC.
- Forecast, observations datasets in-place at CPC.
- FET code ported to CPC, installed, tested.
- FET installed to NWS Web Operations Center (WOC) servers