# Presentation: 129700

## NORTH AMERICAN ENSEMBLE FORECAST SYSTEM

**JOINT CANADIAN-US**
**RESEARCH, DEVELOPMENT, AND IMPLEMENTATION PROJECT**
***Can provide example for THORPEX multi-center ensemble work***

## NORTH AMERICAN ENSEMBLE FORECAST SYSTEM PROJECT

***All NAEFS activities are important to NCEP***
*	**Genuine interest in sharing work and ideas*
*	**NAEFS activities integrated into/with our routine daily work =>*
*Recipe for success?*
**Plan for the morning:**
- General overview of NAEFS Plan			*8:45-9:00*
- Activities and plans related to Initial Operational Capability (IOC) *9-10*
- a)	Data exchange
- 	Communication
- 	Variable list
- Products
- Status report
*BREAK**	***	****	****	****	****	***10:00-10:15*
- Beyond IOC – OPEN ISSUES:			*10:15-10:30*
- Next workshop to coordinate work on bias correction, products, verification
- Future of telecommunication
- Products – Intermediate vs. Final?
- b)	Future ensemble configuration
- Detailed discussions 				*10:30-11:15*
- 	Bias correction
- 	Products
- 	Verification
- Possible areas of expansion – links with THORPEX	*11:15-11:45*
- IOC “ceremony”, wrap-up				*11:45-12:00*

## NORTH AMERICAN ENSEMBLE FORECAST SYSTEM PROJECT

**GOALS:      ****Accelerate improvements in operational weather forecasting**
- 	*through Canadian-US collaboration*
**Seamless (across boundary and in time) suite of products**
- 	*through joint Canadian-US operational ensemble forecast system*
**PARTICIPANTS: ****	****  **Meteorological Service of Canada (CMC, MRB)
- 		  US National Weather Service (NCEP)
**PLANNED ACTIVITIES:  **Ensemble data exchange (June 2004)
- 				   Research and Development -*Statistical post-processing*
*	**	**	*(2003-2007)*	**           -Product development*
*	**	**	**	**	**           **-Verification/Evaluation*
- 				   Operational implementation (2004-2008)
**POTENTIAL PROJECT EXPANSION / LINKS: **
- 		   Shared interest with THORPEX goals of
*	**	**	**Improvements in operational forecasts*
*	**	**	**International collaboration*
- 		   Expand bilateral NAEFS in future
*	**	**	**Entrain broader research community*
*	**	**	**Multi-center / multi-national ensemble system:*
*	**	**Currently available in house: NCEP, MSC, ECMWF, JMA*
***	******	****Aquire in future:**	**	**   FNMOC, UKMET?*

## NAEFS ORGANIZATION

***Meteorological Service of Canada******	******National Weather Service, USA***
***	******	******MSC ******	******	******	******	******	******NWS******	***

- _J.-G. Desmarais (Implementation)_
- Peter Houtekamer (Science)

- _Zoltan Toth (Science)_
- D. Michaud/B. Gordon (Implementatn)

***PROJECT OVERSIGHT***

***JOINT TEAM MEMBERS***

*Meteorological Research Branch MRB*
- Gilbert Brunet 	Herschel Mitchell
- Laurence Wilson
*Canadian Meteorological Center CMC*
- Richard Hogue	Louis Lefaivre
- Richard Verret

*Environmental Modeling Center EMC*
- Lacey Holland	Richard Wobus
- Yuejian Zhu
*NCEP Central Operations NCO*
- TBD
*Hydrometeor. Prediction Center HPC*
- Peter Manousos
*Climate Prediction Center CPC*
- Mike Halpert	David Unger

- Michel Beland, Director, ACSD
- Pierre Dubreil, Director, AEPD

- Louis Uccellini (Director, NCEP)
- D. Perfect (Interntnl. Coordinat., NWS)

***PROJECT CO-LEADERS***

## NAEFS OVERVIEW

- Febr. 2003	MSC – NOAA / NWS high level agreement (Long Beach)
- May 2003	Planning workshop (Montreal)
- Oct 2003	Research, Development, and Implementation Plan complete
- Sept 2004	Initial Operational Capability
- Fall 2004	2nd Workshop (NCEP)?
- July 04 –	3 overlapping18-month R/D & implementation cycles with
- March 08 	Jan 06, Mar 07, Mar 08 implementation dates
- 			*Successively enhanced bias correction, products, verification*
- March 2008	Last operational implementation
***	******	******	****	*

## NAEFS 

**STEP-WISE APPROACH**
- 0)	Initial Oper. Capability – 	Existing products based on other ensemble
- 1)	First Implementation – 	Basic joint forecast system (not comprehens.)
- 2)	Second Implementation -	Refinement (Full system)
- 3)	Final Implementation -	High impact weather enhancements

**NAEFS **
**RESEARCH, DEVELOPMENT, & IMPLEMENTATION PLAN**

## NAEFS 

**RESEARCH, DEVELOPMENT, & IMPLEMENTATION PLAN**

**MAJOR TASKS**
- Exchange ensemble data between 2 centers
- Statistically bias-correct each set of ensemble
- Develop products based on joint ensemble
- Verify joint product suite, Evaluate added value
**COORDINATED EFFORT**
- 	Between Research / development and operational implementation
- 	Between MSC and NWS
- 		***Area of strong common interest between 2 centers, on all levels***
- 		Broaden research scope -		Enhanced quality
- 		Share developmental tasks -		Increased efficiency
- 		Seamless operational suite-		Enhanced product utility
**ROBUST OPERATIONAL SETUP**
**	**Two mirror sites, running same routines provide backup coverage
- 	Single ensemble used in case of communication or computer failures

## NAEFS MAJOR TASKS

**DATA EXCHANGE**
- Identify common set of variables/levels for exchange ~50 fields
- For NCEP data, use GRIB1 with NCEP ensemble PDS extension
- Use native resolution for transfer, convert to common 1x1 (2.5x2.5) grid
- Every 12 hrs, out to 16 days (MSC out to 10 days until later in 2004)
- 		Subset already available on a non-operational basis

## NAEFS MAJOR TASKS – BIAS CORRECTION

**ISSUES**
**Exchange raw or bias-corrected forecasts?**
- 	To ensure 100% backup capabilities =>
- 		Exchange raw data, use same bias-correction at both centers
**Bias-correct before or after merging different ensembles?**
- Sub-components have different biases etc => Calibrate before merging
**Correct univar. prob. distribution functions (pdf) or individual members?**
- 	Users need both – eg, joint probability products (prob hi winds and lo temp)
- 	Correct individual members => pdf falls out free
**Correct for expected value enough?**
- 	No, need to correct for bias in spread => multi-step approach:
- 		a) 	Shift all members
- 		b)	Adjust spread around mean
- 		c)	Reduce temporal variations in spread (if too confident, *Unger*)
**How much training data (forecast – verifying analysis pairs) enough?**
- 	Open research question =>
- 	Need flexible algorithm that can be used either with
- 		Small amount of data – Smooth adjustments to eliminate gross error
- 		Large amount of data – Finer adjustments possible

## NAEFS MAJOR TASKS – PRODUCT DEVELOPMENT

**TYPES OF PRODUCTS**
**A)****	****Joint ensemble** (bias-corrected ensembles merged on model grid)
**B)****	****Anomaly joint ensemble**
-  Express forecast anomalies from reanalysis climatology –
- 						(model grid, easy to ship)
**C)****	****Local joint ensemble forecast** (local, bias-corrected, downscaled)
- 	Add forecast anomaly to observed climatology at
- 		Observational locations or
- 		NDFD high resolution (2.5x2.5 km) grid
**D)****	****Host of products** based on any of 3 choices above
- Gridded, graphical, worded, week 2, etc for
- Intermediate users (forecasters at NCEP, etc)
- End users (automated products at MSC)
- Specialized users
- 	General public
**E)****	****High impact weather products**
- Assess if general procedures above are adequate or can be enhanced for forecasting rare/extreme events

## NAEFS MAJOR TASKS – VERIFICATION

**ISSUES **
- Data sets/archiving – 				Center specific
- Software to compute common set of statistics – 	Shared by 2 centers
- Modular subroutines - common 		Input
- 						Output
- 						Options/parameters
- Verifying against both analysis fields and observations
- Forecast events based on climate or ensemble distribution, or user input
- Benchmarks: climatological, persistence, or alternative forecast systems
- Special product / high impact weather forecast evaluation

## NAEFS - BENEFITS

**Two independently developed systems combined**, using different:
- 	Analysis techniques
- 	Initial perturbations
- 	Models
- 		***Joint ensemble may capture new aspects of forecast uncertainty***
**Procedures / software can be readily applied on other ensembles:**
- 	Possible Multinational expansion linked with THORPEX
- 		ECMWF
- 		JMA
- 		FNMOC
- 		UKMET, etc
- 		***Basis for future multi-center ensemble***
**Collaborative effort**
- Broaden research scope -		Enhanced quality
- Share developmental tasks -		Increased efficiency
- Seamless operational suite -		Enhanced product utility
***	******   ******Framework for future technology infusion (MDL, NOAA Labs, Univs.)***

## IOC

## NAEFS ISSUES – IOC IMPLEMENTATION

**IOC Requirements (by Sept. 2004):**
- 1)	Operationally exchange selected ensemble forecast data between two centers
- Generate separate sets of products based on two ensembles at both centers
**Issues at NCEP:**
- a)	Coordination of variable list with MSC – ***completed***
- 	53 number of fields selected:
- 		Temp, Winds (2), Humidity, Geop. Height at 8 levels
- 			(2/10m except for Z, 1000, 925, 850, 700, 500, 250, 200 hPa)
- SP, MSLP, Top., Precip amount, Pr. types (4), Total cloud cover, PW, Cape, Tmin/Tmax
- 	*Number of variables**	**Current**	**IOC**	**“Final”**	*
*	**NCEP – **	**	**26**	**52**	**53*
*	**MSC - **	**	**17**	**45**	**53*
- 		*Wave products –* 	MSC - 2005? NCEP – Research phase
- 		*Precip type, Tmin,max, CAPE* -	MSC 2005? UNIFY ALGORITHMS
- 		b)	Provide required NCEP ensemble data on NWS ftp server – ***by June 22***
- 		NCEP provides “enspost” type file format (MSC to convert to pgrib at their end)
- 		MSC provides pgrib type file format (NCEP to convert to enspost at their end)
- 		Future – GRIB2 – when? 2-3 yrs?
- c)	Pre-process MSC ensemble data after it is received – ***by June 22***
- 		Convert pgrib into enspost file format
- Generate basic products:
- 	mean, spread – ***by June 22***
- 	PQPF for MSC ensemble – ***under testing***

## Communication issues in Ensemble Data Exchange

- Brent Gordon
- NCEP Central Operations
- NAEFS IOC Presentation
- June 3rd 2004

## NAEFS Data Exchange

- CMC to NCEP
- Using Internet to access data from CMC
- 16 members – 1 cycle per day
- ~ 1 Gb of data per cycle
- NCEP to CMC
- NCEP delivering data to NWS FTP server
- Full data set available 22 June 2004
- 11 members – 2 cycles per day
- ~ 2 Gb of data per cycle

## Data Exchange Challenges

- NCEP use of internet to acquire CMC ensembles not optimal
- Operational reliability issues need to be addressed
- Are occasional internet outages acceptable?
- Future data exchange will most certainly require additional bandwidth
- Funding for NWS/CMC operational network upgrade may be required
- Currently only T-1 access

## Data Exchange Specifications

- Richard Wobus
- Environmental Modeling Center / SAIC
- NAEFS IOC Presentation
- June 3rd 2004

## IOC LIST OF EXCHANGED VARIABLES

**COORDINATION OF VARIABLE LIST WITH MSC** – ***completed***
- 	53 number of fields selected:
- 		Temp, Winds (2), Humidity, Geop. Height at 8 levels
- 			(2/10m except for Z, 1000, 925, 850, 700, 500, 250, 200 hPa)
- SP, MSLP, Top., Precip amount, Pr. types (4), Total cloud cover, PW, Cape, Tmin/Tmax
- 	***Number of variables******	******Current******	******IOC******	****“Final”**	*
***	******NCEP – ******	******	******26******	******52******	****53*
***	******MSC - ******	******	******17******	******45****	**53*
**MISSING VARIABLES:**
- 		*Wave products –* 	MSC - 2005? NCEP – Research phase
- 		*Precip type, Tmin,max, CAPE* -	MSC 2005?
**UNIFY ALGORITHMS FOR PRECIP TYPE, CAPE**

## LIST OF VARIABLES IDENTIFIED FOR ENSEMBLE EXCHANGE BETWEEN CMC  - NCEP

**Blue :  items have been added in prototype script for expanded CMC dataset.**
- Red   : items can be easily added to the expanded dataset via an autoreq for CMC; next implementation period for NCEP
- *    these will be added within 1 month for CMC
- **  these will be added within 2 months for CMC
- Green: items that require further consideration and resources

- Black :  data presently exchanged
**Blue :  items have been added in prototype script for expanded CMC dataset.**
- Red   : items can be easily added to the expanded dataset via an autoreq for CMC; next implementation period for NCEP
- *    these will be added within 1 month for CMC
- **  these will be added within 2 months for CMC
- Green: items that require further consideration and resources

**LIST OF VARIABLES IDENTIFIED FOR ENSEMBLE EXCHANGE ****BETWEEN CMC  - NCEP**

## Height

**Blue :  items have been added in prototype script for expanded CMC dataset.**
- Red   : items can be easily added to the expanded dataset via an autoreq for CMC; next implementation period for NCEP
- *    these will be added within 1 month for CMC
- **  these will be added within 2 months for CMC
- Green: items that require further consideration and resources

- Temperature
- Humidity
- Wind
- Other
- Summary “Appendix 5”

**LIST OF VARIABLES FOR ENSEMBLE EXCHANGE ****BETWEEN CMC  - NCEP**

## Enspost and Ensstat data exchange - height

| Variable | Present from NCEP | June 2004 from NCEP | June 2004 from CMC | Future from NCEP and CMC |
| --- | --- | --- | --- | --- |
| z200 |  | X | X | X |
| z250 | X | X | X | X |
| z500 | X | X | X | X |
| z700 | X | X | X | X |
| z850 |  | X | X | X |
| z925 | X | X | X | X |
| z1000 | X | X | X | X |
| zsfc |  | X | X | X |

## Enspost and Ensstat data exchange - Temperature

| Variable | Present from NCEP | June 2004 from NCEP | June 2004 from CMC | Future from NCEP and CMC |
| --- | --- | --- | --- | --- |
| t200 |  | X | X | X |
| t250 | X | X | X | X |
| t500 | X | X | X | X |
| t700 | X | X | X | X |
| t850 | X | X | X | X |
| t925 |  | X | X | X |
| t1000 | X | X | X | X |
| t2m | X | X | X | X |
| tmin 2m | X | X | X | X |
| tmax 2m | X | X | X | X |

## Enspost and Ensstat data exchange – humidity

| Variable | Present from NCEP | June 2004 from NCEP | June 2004 from CMC | Future from NCEP and CMC |
| --- | --- | --- | --- | --- |
| rh200 |  | X | X | X |
| rh250 |  | X | X | X |
| rh500 |  | X | X | X |
| rh700 | X | X | X | X |
| rh850 |  | X | X | X |
| rh925 |  | X | X | X |
| rh1000 |  | X | X | X |
| rh2m |  | X | X | X |

## Enspost and Ensstat data exchange - winds

| Variable | Present from NCEP | June 2004 from NCEP | June 2004 from CMC | Future from NCEP and CMC |
| --- | --- | --- | --- | --- |
| u,v 200 |  | X | X | X |
| u,v 250 | X | X | X | X |
| u,v 500 | X | X | X | X |
| u,v 700 |  | X | X | X |
| u,v 850 | X | X | X | X |
| u,v 925 |  | X | X | X |
| u,v 1000 |  | X | X | X |
| u,v 2m | X | X | X | X |

| Variable | Present from NCEP | June 2004 from NCEP | June 2004 from CMC | Future from NCEP and CMC |
| --- | --- | --- | --- | --- |
| pmsl | X | X | X | X |
| psfc |  | X | X | X |
| prcp | X | X | X | X |
| prcp type | X | X |  | X |
| pwat |  | X | X | X |
| cape |  | X |  | X |
| tot cld cov |  | X | X | X |
| wave ht |  |  |  | X |

- Enspost and Ensstat data exchange – other variables

## Derived products

- Yuejian Zhu
- Environmental Modeling Center
- NAEFS IOC Presentation
- June 3rd 2004

## Derived Products - grids

- Ensemble Mean and Spread (**NAEFS-IOC**)
- Probabilistic Forecasts
- Probabilistic Quantitative Precipitation Forecast (PQPF) (**NAEFS-IOC**)
- Precipitation type forecast (PQRF,PQSF,PQFF & PQIF) (**NCEP-OPR**, future NAEFS)
- Calibrated PQPF (NCEP-OPR, future NAEFS)
- Relative Measure of Predictability (**EMC-EXP**)
- Verifications (deterministic and probabilistic)
- Against global analysis (**EMC-EXP**)
- Against observation (Work started)
- Special products generated locally by NCEP Service Centers (to be discussed later)

## Derived Products - Graphics (NCEP-para)

- Ensemble Mean and Spread (Tim Marchok)
- Probabilistic Forecasts
- Probabilistic Quantitative Precipitation Forecast (PQPF)
- Precipitation type (PQPF, PQRF,PQSF & PQF+IF)
- Calibrated QPF and PQPF
- Relative Measure of Predictability (RMOP)
- Spaghetti diagrams (Bill Bua)
- Cyclone tracks (Tim Marchok)
- Verifications (against analysis and observation)

## PQPF example for NCEP, CMC and ECMWF

**PQPF example for NCEP, CMC and ECMWF**

## NCEP PQPTF example

**NCEP PQPTF example**

## NCEP Calibrated PQPF example

**NCEP Calibrated PQPF example**

## By Bill Bua

## By Tim Marchok

## Slide 35

##            1. By using equal climatological bins (e.g. 10 bins, each grid points)2. Counts of ensemble members agree with ensemble mean, (same bin)3. Construct n+1 probabilities for n ensemble members from (2).3. Regional (NH, weighted) Normalized Accumulated Probabilities (n+1)4. Calculate RMOP based on (3), but 30-d decaying average.5. Verification information (blue numbers):  historical average (reliability)

## By Tim Marchok

## Schedule (FY)

## Performance Parameters

- The NWS portion of the US-Canadian North American Global Ensemble Forecast System Development and Implementation.

## BEYOND IOC

## BEYOND IOC – OPEN ISSUES

**1)****	****NEED 2****ND**** WORKSHOP**
- Jump start detailed planning and coordination in areas of
- Bias correction
- Product development
- Verification
- Fall 2004 at NCEP?
**FUTURE TELECOMMUNICATION NEEDS**
- Will current ftp process be adequate in future?
- Increased volume of data due to higher resolution & more members
- Wide-range operational use may demand more reliability?
- Switch to GRIB2
- Factor of 3 reduction in data volume due to more efficient packing
- WMO standard, preferred for future multi-center data exchange
- Advance planning needed, implement in 2-3 yrs?
**COMMON OR DIFFERENT PRODUCT SUITE?**
- Different emphasis on 2 sides:
- MSC – Fully automated forecast process, SCRIBE
- Automated products for selected sites, final products?
- NWS – Larger role for human forecaster, IFPS
-  Intermediate guidance products on NDFD grid?

## BEYOND IOC – OPEN ISSUES - 2

**GLOBAL ENSEMBLE FORECAST SYSTEMS OPERATIONAL AT CMC AND NCEP**

**BEYOND IOC – OPEN ISSUES - 2**

**ENSEMBLE CONFIGURATION**
- What horizontal/vertical resolution; How many members?
- Formal configuration requirements, or quality driven choices?

## DETAILED DISCUSSIONS

- BIAS CORRECTION
- PRODUCTS
- VERIFICATION

## BASED ON2ND ENSEMBLE USER WORKSHOPMay 18-20 2004, NCEP  DRAFT RECOMMENDATIONS Based on presentations and working group discussionsJune 1 2004

## WORKING GROUP PARTICIPANTS (26)

**CONFIGURATION**
*Co-leaders:**** ***Jun Du and Mozheng Wei
*Participants: *Rick Knabb, Richard Wobus, Ed O’Lenic, Dingchen Hou

**STATISTICAL POST-PROCESSING**
*Co-leaders:* Paul Dallavalle & Zoltan Toth
*Participants: *Keith Brill, Andy Lough, DJ Seo, David Unger

**DATA ACCESS**
*Co-leaders: *Yuejian Zhu and David Michaud
*Participants: David Bright, Minh Nguy, Kathryn Hughes*

**PRODUCTS & TRAINING**
*Co-leaders:* Jeff McQueen and Pete Manousos
*Participants: Paul Stokols, Fred Mosher, Paul Janish, Linnae Neyman, Bill Bua, Joe Sienkiewicz, Binbin Zhou*

***ADDITIONAL WORKSHOP PARTICIPANTS (15)***
- Steve Tracton, Mike Halpert, Brian Gockel, Brent Gordon, Mark Antolik, Barbara Strunder, Andrew Loughe, Michael Graf, Dave Plummer, Steve Schotz, Jon Mittelstadt, Malaquias Pena, Glen Zolph, Steve Lord, David Caldwell

## 2nd NCEP Ensemble User Workshop SUMMARY RECOMMENDATIONS

**OVERALL** - 	Enhance coordination of ensemble-related efforts
- Establish ensemble product working group
- Continue with monthly Predictability meetings
- Hold Ensemble User Workshops (part of reestablished SOO workshops)
**CONFIGURATION**
*	****Global ensemble:****	**	*Implement hurricane relocation for perturbed initial conditions
- 				Continue efforts to build multi-center ensemble
*	****Regional (SREF) ensemble:****	*Ensemble run should be coupled closer with hires control (same initial time)
- 				Run 4 cycles per day
**DATA ACCESS**
- Provide access to all ensemble data (including members)
- Facilitate user controlled access to data (e.g. NOMAD, on demand, not on rigid schedule)
**STATISTICAL POST-PROCESSING (BIAS CORRECTION)**
- Develop techniques for two-stage statistical post-processing
- Operationally implement post-processing techniques
**PRODUCTS**
- Develop a software toolbox for interrogating ensemble data
- Establish central/local operational product generation suites
**VERIFICATION **
- Design & develop unified and modular ensemble/probabilistic verification framework
**TRAINING**
- Establish NWS formal ensemble training requirements
- Contribute to Ensemble Training Workshops, international activities (AMS, WMO), etc

## ENSEMBLE STATISTICAL POSTPROCESSING - CURRENT STATUS

**NWP models, ensemble formation are imperfect**
- Known model/ensemble problems addressed at their source
- No “perfect” solution exists, or is expected to emerge
- Systematic errors remain and cause biases in
- 1st, 2nd moments of ensemble distribution
- Spatio-temporal variations in 2nd moment
- Tails of distributions
**No comprehensive operational post-processing in place**
- MOS applied on individual members (global ensemble, MDL)
- QPF calibration of 1st moment (global ensemble, EMC & CPC)
- Week 2 calibration with frozen system (global ensemble, CDC)
**Issues:**
- Users need bias-free ensemble guidance products
- Bias-corrected ensemble members must be consistent with verification data
- Algorithms must be relatively cheap & flexible for operational applications
- Post-process on model grid first, then “downscale” to NDFD grid / observs?
- Level of “correctible” details depends on
- Bias signal vs. random error noise ratio
- Sample size of available forecast/observation training data pairs
- Relatively small sample for short-med. ranges – Capture regime dependent bias?
- Much larger for extended ranges – Capture climatological bias via frozen system?

## Develop techniques for two-stage statistical post-processing:
1) Assess and mitigate biases on model grid with respect to analysis fields
Feedback to model / ensemble development
1st moment correction based on: Time mean error; Cumulative distributions 
2nd moment correction based on: Time mean ratio of ens mean error & spread
Post-processed forecasts bias corrected with respect to reanalysis fields
Generate anomaly forecasts using global/regional reanalysis climatology
2) Downscale bias-corrected fcsts from model grid to NDFD/observatn locations
“Smart” interpolator for bias correction and variance generation on fine scales
Multiple regression (MOS); Bayesian methods; Kalman Filtering; Neural nets
Apply downscaling methods on bias-corrected fields (no lead time dependence)
Use large reanalysis and corresponding observational data base (&/or NDFD analysis fields)
To describe ensemble-based pdf forecasts, use 3-parameter distributions
Test two methods, find best fitting analytic distribution (out of ~25 candidates)
Simple method:	Fit actual ensemble data
Kernel approach: 	Find best fit to climate data, then apply it on each member w/weight
Operationally implement post-processing techniques
Apply basic bias-correction techniques centrally (NCO) to serve wide user base
Post-process all variables used from the ensemble (first model, then derived variables)
Disseminate bias-corrected forecasts on lowres ensemble model grid
Save disc and bandwidth resources
Keep raw forecast fields also accessible for special user processing needs
Use additional post-processing (if any) locally to address special needs, eg:
Hurricane forecasting

- 1) Assess and mitigate biases on model grid with respect to analysis fields
- Feedback to model / ensemble development
- 1st moment correction based on: Time mean error; Cumulative distributions
- 2nd moment correction based on: Time mean ratio of ens mean error & spread
- Post-processed forecasts bias corrected with respect to reanalysis fields
- Generate anomaly forecasts using global/regional reanalysis climatology
- 2) Downscale bias-corrected fcsts from model grid to NDFD/observatn locations
- “Smart” interpolator for bias correction and variance generation on fine scales
- Multiple regression (MOS); Bayesian methods; Kalman Filtering; Neural nets
- Apply downscaling methods on bias-corrected fields (no lead time dependence)
- Use large reanalysis and corresponding observational data base (&/or NDFD analysis fields)
- To describe ensemble-based pdf forecasts, use 3-parameter distributions
- Test two methods, find best fitting analytic distribution (out of ~25 candidates)
- Simple method:	Fit actual ensemble data
- Kernel approach: 	Find best fit to climate data, then apply it on each member w/weight
- Operationally implement post-processing techniques
- Apply basic bias-correction techniques centrally (NCO) to serve wide user base
- Post-process all variables used from the ensemble (first model, then derived variables)
- Disseminate bias-corrected forecasts on lowres ensemble model grid
- Save disc and bandwidth resources
- Keep raw forecast fields also accessible for special user processing needs
- Use additional post-processing (if any) locally to address special needs, eg:
- Hurricane forecasting

**ENSEMBLE STATISTICAL POSTPROCESSING - RECOMMENDATION**

## ENSEMBLE PRODUCTS - CURRENT STATUS

**Product development software**
- Some functionalities exist
- Scattered around different developers/platforms/users
- NCO operations
- NAWIPS official build
- NAWIPS development by NCEP SOOs
- AWIPS
- Other platforms
**Products generated centrally by**
- NCO	Limited number of gridded products (operational)
- EMC	Additional set of gridded and web-based products (non-operational)
**Issues:**
- Lack of standard/common software toolbox for ensembles
- Missing functionalities
- Multiple software versions of existing functionalities
- Duplication of efforts
- Lack of comprehensive, well designed set of products
- Non-standard set of products/displays (global vs. regional ensembles, etc)
- NAWIPS, AWIPS requires access to products (web not enough)
- Need for operationally generated and supported web product suite

## ENSEMBLE PRODUCTS - RECOMMENDATIONS

- Develop a software toolbox for interrogating ensemble data
- Establish development team - NCO, EMC, NCEP Service Center experts
- Compile list of required functionalities – See attached list
- Develop standard software package (subroutines) for each functionality
- Work in NAWIPS framework
- Ensure software (subroutines) are portable to different platforms
- Ensure batch and on demand processing capabilities
- Provide interactive processing/display capability where needed
- Offer subroutines for use by AWIPS and broader inter/national community
- Consider WRF, NAEFS, THORPEX applications
- Establish operational/local product generation suites
- Use standard software toolbox for product generation
- Identify list of products – See template on next page
- Type of product generation based on typical usage:
- Every day	- Generate centrally (NCO), produce multiple file formats
- Occasionally	- On demand (NCEP Service Centers)
- Interactively	- On screen manipulation (NAWIPS)
- Distribute centrally generated products within NAWIPS, AWIPS
- Set up and maintain operational NCEP ensemble product web page
- Post products on web page for use by broader community
- Provide limited interactive query tools if desired (example within NOMAD)

## ENSEMBLE PRODUCTS - FUNCTIONALITIES

|  | FUNCTIONALITY | CENTRALLY GENERATED | LOCALLY GENERATED |
| --- | --- | --- | --- |
| 1 | Mean of selected members |  |  |
| 2 | Spread of selected members |  |  |
| 3 | Median of selected values |  |  |
| 4 | Lowest value in selected members |  |  |
| 5 | Highest value in selected members |  |  |
| 6 | Range between lowest and highest values |  |  |
| 7 | Univariate exceedance probabilities for a selectable threshold value  |  |  |
| 8 | Multivariate (up to 5) exceedance probabilities for a selectable threshold value  |  |  |
| 9 | Forecast value associated with selected univariate percentile value  |  |  |
| 10 | Tracking center of maxima or minima in a gridded field (eg – low pressure centers)  |  |  |
| 11 | Objective grouping of members |  |  |
| 12 | Plot Frequency / Fitted probability density function at selected location/time (lower priority)  |  |  |
| 13 | Plot Frequency / Fitted probability density as a function of forecast lead time, at selected location (lower priority)  |  |  |

- For each functionality, NCEP Service Centers provide list of variables/levels for which central/local generation of products is needed: *MSLP, Z,T,U,V,RH, etc, at 925,850,700,500, 400, 300, 250, 100, etc hPa*

## ENSEMBLE VERIFICATION – CURRENT STATUSFor lack of time, this topic was not discussed at the workshop

- Global ensemble verification package used since 1995
- Comprehensive verification stats computed against analysis fields
- Inter-comparison with other NWP centers
- Regional (SREF) verification package
- Basic measures computed routinely since 1998
- Probabilistic measures being developed independently from global ensemble
- Issues
- Need to unify computation of global – regional ensemble verification measures
- Unified framework must facilitate wide-scale national/international collaboration:
- North American Ensemble Forecast System (collaboration with Met. Service Canada)
- THORPEX International Research Program
- WRF meso-scale ensemble developmental and operational activities
-  Facilitate wider community input in further development/enhancements
- How to establish basis for collaboration with NCAR, statistical community, etc

## ENSEMBLE VERIFICATION - RECOMMENDATIONS

- Design unified and modular ensemble/probabilistic verification framework
- Data handling/storage
- Use standard WMO file formats as ensemble data input
- Allow non-standardized user/site specific procedures
- Computation of statistics
- Establish required software functionalities (scripts) and verification statistics (codes)
- Jointly develop and share scripts/subroutines with standard input/output fields
- Improvements to common infrastructure benefit all
- Comparable scientific results, independent of investigators
- Access/display of output statistics
- Explore if standard output file format(s) feasible? Use text or FVSB-type files?
- Develop/adapt display software for interactive interrogation of output statistics
- Examples: FVS display system; FSL approach to WRF verification
- Develop and implement new verification framework
- Utilize existing software and infrastructure where possible
- Direct all internal ensemble-related verification efforts toward new framework
- Share work with interested collaborators
- Meteorological Service of Canada (subroutines, L. Wilson and colleagues)
- FSL (display tools, A. Laugh)
- Make new software available to national/international community
- Coordinate further development with wider community (WMO, etc input)

## ENSEMBLE VERIFICATION – DESIGN SPECIFICATIONS

**Compute statistics selected from list of available**
- Point-wise measures, including:
- RMS, PAC for individual members, mean, median
- Measures of reliability (Talagrand, spread vs. error, reliability components of Brier, RPSS, etc)
- Measures of resolution (ROC, info content, resol. comps. of BSS, RPSS, potential econ.value, etc)
- Combined measures of reliability/resolution (BSS, RPSS, etc)
- Multivariate statistics (e.g., PECA, etc)
- Variables & lead times –make all available that are used from ensemble
**Aggregate statistics as chosen in time and space**
- Select time periods
- Select spatial domain (pre-designed or user specified areas)
**Verify against observational data or analysis fields**
- Scripts running verification codes should handle verification data issues
- Use same subroutines to compute statistics in either case
- Account for effect of observational/analysis uncertainty?
**Define forecast/verification events by either**
- Observed/analyzed climatology, e.g., 10 percentile thresholds in climate distribution
- Automatically compute thresholds for forecast values
- User specified thresholds – automatically compute corresponding climate percentiles
- Ensemble members (like in Talagrand stats) – compute climate percentiles
**Facilitate the use of benchmarks**:
- Climatology, persistence, or specified prior forecast data set
**Prioritize and find balance between **
- Flexibility vs. complexity; operational vs. research use, etc

## POTENTIAL FUTURE EXPANSIONS

- NEW AREAS OF COMMON INTEREST IN RESEARCH/DEVELOPMENT
- LINKS WITH THORPEX

## NAEFS 

**FUTURE JOINT RESEARCH OPPORTUNITIES**

**Ensemble configuration - **
- 		Model resolution vs. membership, etc
**Representing model errors in ensemble forecasting** –
- 		High priority research area, collaboration possible
**Initial ensemble perturbations** –
- 		Compare 2 existing systems, may improve both
**Ensemble forecasting on different scales**:
- 	*Regional ensemble forecasting:*   No activities at MSC, maybe in 2 yrs
*	**3-6 weeks – seasonal:*	        Opportunities for research collaboration

## NAEFS 

**LINKS WITH THORPEX**

- THORPEX TIP adapted multi-center ensemble concept
- 	Ensembles collected and processed at multiple sites
- 	Products made available internationally
- NAEFS plan can serve as a draft for “blueprint” of multicenter concept
- 	MSC, NCEP should play proactive role
- 		Careful considerations for operational application
- 		Model that (we hope) will work
- 	Benefits from international collaboration
- 	Service to underdeveloped countries
- THORPEX TIP calls for IPY collaboration
- 	IPY of great interest to both countries
- 	Opportunity for joint IPY-related activities
- 	US strawperson proposal (Parsons, Shapiro, Toth)
- Other promising areas under THORPEX TIP?

## PROPOSAL FOR IPY-RELATED THORPEX FIELD CAMPAIGN

**International Polar Year (IPY):**
- 	**Multi- and interdisciplinary international research experiment in 2007-2008**
- 	*Study areas of strongest climate change impact*
*	**Research in both polar regions*
*	**Strong links to the rest of the globe*
**THORPEX – Global Atmospheric Research Program (GARP): **
- 	**Accelerate improvements in skill/utility of 1-14 day weather forecasts**
- Long-term (10-yrs) research program in areas of:
- 	Observing system, data assimilation, numerical modeling/ensemble, socioec. appl.
- Strong link with operational Numerical Weather Prediction (NWP) centers
- International program under WMO
*Planning initiated with discussions about North Pacific experiment =>*
**Opportunities for IPY - THORPEX Collaboration**
- 	**Joint THORPEX-IPY Observing period –**
- 		Enhanced observational coverage for both programs
**	****Improved weather forecasts for IPY activities**
- 	**Scientific investigations:**
- 		Link between weather and climate processes
- 		Mid-latitude – Polar interactions

## Opportunities for THORPEX:

***2-MONTH FIELD PROGRAM DURING IPY***
***Joint THORPEX – IPY Observing Period, Winter of 2007/08 ***
**Enhance atmospheric observations in NW Pacific** - *Contributing to IPY activities*
- 	Manned and unmanned aircraft, driftsonde, satellite, etc
- 	Extension of operational NWS Winter Storm Recon coverage (northeast Pacific)
- Targeted to improve Alaskan (and Northern Canadian) 2-3 day forecasts
- Study mid-latitude – polar interaction on daily time scale
**Utilize enhanced IPY polar observing system in NWP**** – ***Advantages for THORPEX*
**	**Ensure real-time accessibility of data (for NWP centers, through GTS transmission)
**	**Explore targeted use of IPY data on 2-3 day time scale over NA (cold air outbreaks, etc)
- 	Consider special enhancement of IPY data if needed
**Evaluate effect of enhanced observing system on forecasts** – *Mutual benefits*
- Study combined effect of North Pacific (NP-TREC) & polar region (IPY) observations
- 	2-3 days – Polar regions of NA; 3-14 days – NA, NH, Global domains
***PLANNING:******	******	***a) Interface with IPY - International THORPEX coordination
- 		b) Develop detailed US plan – Coordinate within NA
- 		c) Start scientific work (eg, OSSE) as soon as possible
- 		d) NP-TOST for testing new components of observing system (2006)

**Opportunities for THORPEX:**
- Scientific collaboration on time scales of 	weather/climate interface
- Observing system enhancements over poles

**Benefits for IPY:**
- Link to mid-latitude weather processes 	(science and organizational)
- Improved targeted weather forecasts

***Ample time for planning coordinated field program - Possible joint funding opportunities***

## IOC CEREMONY

- Coinciding with 2nd NAEFS Workshop in Fall 2004?
- At opening of workshop?

## BACKGROUND

## North American Global Ensemble Forecast System

**(D/M/I)**

- Prime Contractors:  NOAA/NCEP/EMC

- Director:  Louis W. Uccellini
- PM:  Stephen J. Lord

_**Performance Parameters**_

- Stephen Lord/W/NP2/May 31, 2004

**                 **

**G**

_**Key Issues/Risks**_

_**Budget/Funding $K **_

_**Schedule**_** (FY)**

**Program Is Executable**

**G**

**G**

- NP-3

- The NWS portion of the US-Canadian North American Global Ensemble Forecast System Development and Implementation.

- None

**G**