# Presentation: 182553

## COMPLETING THE FORECAST:ASSESSING AND COMMUNICATING FORECAST UNCERTAINTY

- Zoltan Toth
- Environmental Modeling Center
- NOAA/NWS/NCEP
- Acknowledgements:
- NRC Report; Louis Uccellini, Steve Lord & Ensemble Team
- http://wwwt.emc.ncep.noaa.gov/gmb/ens/index.html

## OUTLINE / SUMMARY

- SCIENTIFIC NEED TO ASSESS/COMMUN. UNCERTAINTY
- Science of prediction		CHAOS
- Science of applications		UNIQUE USER NEEDS
- TECHNICAL READINESS
- Ensemble forecasting (push)
- OPERATIONAL REQUIREMENTS
- Need to be formalized for forecast producers
- USER ACCEPTANCE / DEMAND
- Needed to instigate / force changes (pull)
- RECENT DEVELOPMENTS
- NRC Report, NFUSE, NAEFS, THORPEX, NUOPC
- Role of OFCM?

## SCIENTIFIC BACKGROUND:

**WEATHER FORECASTS ARE UNCERTAIN**

*Buizza 2002*

**USER REQUIREMENTS:****PROBABILISTIC FORECAST INFORMATION IS CRITICAL**

## WHY CHANGE IS NEEDED?

- They admittedly want minimal or no uncertainty in forecasts
- Distinction between no uncertainty in the forecast, vs. not talking about it
- Forecast uncertainty cannot be arbitrarily reduced
- Despite major ongoing & continuing efforts, they persist forever
- Chaotic nature of atmosphere - land surface – ocean coupled system + initial/model errors
- Level of uncertainty is determined by nature and level of sophistication in forecast system
- Forecast uncertainty can be ignored though
- Negative consequence on informed users
- Not able to prepare for all possible outcomes
- Assumes a certain scenario and remains vulnerable to others
- Possibly serious loss in social/economic value of forecast information
- Why forecasters (should) care about forecast uncertainty?
- Imperfect forecasts are consistent w. observations (reliable) only if in prob format
- If in other format, must be brought into probabilistic format through
- Verification / bias correction

**WHY CHANGE IS NEEDED?**

## PROPOSED CHANGE

- Incorporate assessment and communication of uncertainty in forecast process
- Is it a **major change in course** of “Weather Ship”?
- Ie, abandon course of ever improving single forecast scenario (expected value)?
- No – Expand, not abandon
- Keep improving fidelity of forecasts, PLUS
- Add new dimension
- Capture other possible scenarios – ensemble forecasting
- Use a flotilla, instead of one ship, in exploring nature
- Existing activities are subset of expanded forecast process
- Single value forecast is expected value of full probability distribution
- Can keep serving forecasts in old format to users who prefer that

**PROPOSED CHANGE**

## Single forecast (driven by GFS winds) example for drifting virtual ice floe

- Single forecast (driven by GFS winds) example for drifting virtual ice floe

*Bob Grumbine, EMC*

- Initial position

- 7 September 2006

## Ensemble forecast for drifting ice floe for same case

**Ensemble forecast** for drifting ice floe for same case

*Bob Grumbine, EMC*

- Initial position

## Most likely forecast for drifting ice floe for same case

- Most likely forecast for drifting ice floe for same case

*Bob Grumbine, EMC*

- Initial position

## PROPAGATING FORECAST UNCERTAINTY

- z

*Single value*

*Distribution*

- Ensemble Forecasting:
- Central role – bringing the pieces together

**PROPAGATING FORECAST UNCERTAINTY**

## HOW CAN IT BE DONE? NEW PARADIGM

- Expand forecasting with new dimension of uncertainty
- Multiple scenarios (in place of single scenario)
- Provides best forecast estimate for both expected value (as before) and uncertainty (new)
- Unified scientific, technological, human approach
- Sharing resources across NWS & NOAA
- Ensemble is centerpiece both symbolically and figuratively in forecast process
- Ensembles act as a glue & two-way information channel
- Observing system, data assimilation, numerical modeling
- ENSEMBLES
- Statistical post-processing, product generation, decision making
- Design, develop, & implement missing components of new forecast process
- Gradual, measured steps
- Basic capability - Short-term, 2-3 yrs, leading to
- Full implementation - Long-term, 5-10 yrs

**HOW CAN IT BE DONE? NEW PARADIGM**

## REVISE OPERATIONAL FORECASING REQUIREMENTS

- For each forecast application:
- Replace single value/categorical format with probabilistic format as primary requirement
- Revise/supplement corresponding performance measures (GPRA)
- Essential for orderly transition from traditional to new forecast process
- NWS is requirement-driven organization
- Without clear new requirements, process is doomed
- Phased implementation schedule in consultation with responsible offices
- Allows orderly transition to new requirements

**REVISE OPERATIONAL FORECASING REQUIREMENTS**

## ADVANTAGES OF PROBABILISTIC FORMAT

**Old paradigm**
- Convoluted forecaster-user decision process
- User expects forecaster to make decision for them in presence of uncertainty
- “Will it rain?” – “80%” – “But tell me, will it rain?”
**New paradigm**
- Forecaster and user decision processes enhanced and better linked
- Allows forecasters to capture all knowledge about future conditions
- Provision of information related to multiple decision levels in probabilistic format critical
- Provider helps interpret probabilistic info & and modify user decision process if needed
- Option to continue providing single value or other limited info until user ready
- Allows users to decide about most beneficial course of action given all possibilities
- Proper use of probability or other uncertainty information needed - Training
- User requests critical weather forecast info depending on their sensitivity

**ADVANTAGES OF PROBABILISTIC FORMAT**

## ENSEMBLES AND THE RESEARCH COMMUNITY

**SOCIOECON.**

**ENSEMBLES AND THE RESEARCH COMMUNITY**
- LINKED THROUGH THORPEX – MAJOR INTERNATIONAL RESEARCH PROGRAM
***GOAL: Accelerate improvements of high impact weather forecasts***

**SYSTEM**

- CLIMATE FORECASTING / CTB

***INTEGRATED**** DATA ASSIMILATION & FORECASTING*

***Days 15-60***

- NWS OPERATIONS

- GLOBAL INTERACTIVE FORECAST SYSTEM (GIFS)

***GLOBAL OPERATIONAL***

***TEST CENTER***

***GLOBAL OPERATIONAL***

***TEST CENTER***

***MODEL ****ERRORS & HIGH IMPACT MODELING*

## NORTH AMERICAN ENSEMBLE FORECAST SYSTEM (NAEFS)

**NORTH AMERICAN ENSEMBLE FORECAST SYSTEM ****(NAEFS)**

***International project to produce operational multi-center ensemble products***
- Combines global ensemble forecasts from Canada & USA
-  40 members per cycle, 2 cycles per day from MSC & NWS
- 6-hourly output frequency
- Forecasts out to 16 days
- Generates products for
- Weather forecasters
- E.g., NCEP Service Centers (US NWS)
- Specialized users
- E.g., hydrologic applications in all three countries
- End users
- E.g., forecasts for public distribution in Canada (MSC) and Mexico (NMSM)
- Operational outlet for THORPEX research using TIGGE archive
- Prototype ensemble component of THORPEX Global Interactive Forecast System (GIFS)

## Fcst: 24hr Ensemble Mean  & Bias Before/After Downscaling 10%

- Before

- Before

- After

-  Left top: operational ens. mean and its bias wrt RTMA
-  Right top: bias corrected ens. mean and its bias wrt RTMA
-  Left bottom: bias corrected & downscaled ( 10% ) ens. mean and its bias wrt RTMA** **
- After Downscaling
-  Bias reduced, especially high topography areas
-  More detailed forecast information

## Fcst: 24hr Ensemble Mean  & Bias Before/After Downscaling 10%

- Before

- Before

- After

## 2m Temperature: Continuous Ranked Probability Score (CRPS) 

- Average for 20070212 to 20070404

- Preliminary results:
***Major improvement in skill of fine-scale forecasts:*** Downscaled & bias-corrected ensemble forecasts have significant improvements compared with raw & calibrated forecast for all lead time (downscaled 5+day forecast as skillful as raw 6-hr forecast)
-  10% weighting is better than 2% and 5% weighting in short range. ~30% improvement with 10% weighting for d0-d4. The 2%, 5% and 10% weighting curves are close for long range. Will add more high weights for comparison.
**Limitation:  **
** ****small samples**
** ****more samples needed  **

- _Before downscaling_

- _After downscaling_

## RECENT DEVELOPMENTS

- Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts
- Recommendations are evaluated by NWS/NOAA
- NWS Forecast Uncertainty Service Evolution (NFUSE) team
- Overall coordination of related planning work
- National Unified Operational Prediction Capability (NUOPC)
- Triagency (NOAA, US Navy, US Air Force project
- Multicenter ensemble forecasting
- Planning stage
- US THORPEX research activities
- NSF, NOAA, ONR, NASA coordination of research on
- High impact weather forecasting
- US THORPEX Project Office
- Would OFCM be interested to contribute?

**RECENT DEVELOPMENTS**

## OUTLINE / SUMMARY

- SCIENTIFIC NEED TO ASSESS/COMMUN. UNCERTAINTY
- Science of prediction		CHAOS
- Science of applications		UNIQUE USER NEEDS
- TECHNICAL READINESS
- Ensemble forecasting (push)
- OPERATIONAL REQUIREMENTS
- Need to be formalized for forecast producers
- USER ACCEPTANCE / DEMAND
- Needed to instigate / force changes (pull)
- RECENT DEVELOPMENTS
- NRC Report, NFUSE, NAEFS, THORPEX, NUOPC
- Role of OFCM?