# Presentation: 830130

## ENSEMBLE FORECASTING

- Zoltan Toth
- Ensemble & Probabilistic Guidance Team
- Environmental Modeling Center

## OUTLINE

- User requirements / Decision support
- Ensemble system upgrade plans
- GEFS
- SREF
- Ensemble processing plans
- Bias correction
- Downscaling
- Products

## USER REQUIREMENTS - SIMPLIFIED

**Applications affected by weather**
- Must consider information on weather to
- Minimize losses due to adverse weather
**Optimal user decision threshold** equals
- Probability of adverse weather exceeding
- Cost / loss ratio of decision situation (simplified decision theory)
**Probability of weather events** must be provided
- Only option in past, based on error statistics of single value forecasts
- Eg, MOS POP
- Now can be based on ensemble
- Users act when forecast probability exceeds their cost/loss ratio
**Advantages**
- Limited set of products (eg, 10 / 50 / 90 percentile forecast) doable “today”
**Problems**
- Proliferation of number of products
- Different variables, probability / weather element thresholds, joint probabilities
- Limited usage
- Downstream applications severely limited (eg, wave, streamflow, etc, ensembles not possible)

## USER REQUIREMENTS - ADVANCED

- Some (many?)** applications affected by**
**Joint probabilities**
- Eg, probability of significant convection in vicinity of 2 specific hubs between 3-6pm
- Many variables, many probability / critical value decision thresholds
- Access to multiple (n) plausible weather scenarios critical
- Statistically reliable **ensemble weather data**
***6D-Cube*** - space (3D) + time + variables + ensemble
- Expanded NDFD – future official NWS weather / climate / water forecast database
- Develop **user application model** simulating optimal weather related operations including
- Assuming weather is known – What are actions / costs / benefits?
**Run application model n x n times** with
- Weather scenario from each ensemble member using
- User operations (oi) optimized for each ensemble member (ei)
- Take **ensemble mean of economic outcome** (costs + losses) for each set of user procedures
- Choose operating procedures (oi) minimizing costs + losses in expected (ensemble mean) sense
**Advantages**
- General approach to optimizing weather related decisions
- Focus on optimization of 2 problems
- Forecast ensemble (including statistical reliability)
- User application modeling
**Challenge**
- Requires
- Storage / telecom bandwidth
- Smart sub-setting & interrogation tools
*Can derive any weather related information including joint probabilities*

## RESPONSIBILITIES OF ENSEMBLE TEAM

*Assess, model, communicate uncertainty in numerical forecasts*
- Represent uncertainty in numerical forecasting
- Tasks
- Design, implement, maintain, and continuously improve ensemble systems
- Topics
- Initial value related uncertainty
- Model related forecast uncertainty
- Ensemble systems
- Global – NAEFS / GEFS
- Regional – SREF
- Climate – Contributions to future CFS configuration
- Statistical correction of ensemble forecasts
- Tasks
- Correct for systematic errors on model grid
- Downscale information to fine resolution grid (NDFD)
- Combine all forecast info into single ensemble/probabilistic guidance
- Probabilistic product generation / user applications
- Contribute to design of probabilistic products
- Support use of ensembles by
- Internal users (NCEP Service Center, WFO, etc forecasters)
- External users (research, development, and applications

## NEXT GEFS IMPLEMENTATION (2009)

**Increase horizontal resolution** from T126 to T190
- 4 cycles per day, 20+1 members per cycle
- Up to 384 hours (16 days)
- Add **stochastic perturbation scheme** to account for random model errors
- Increased ensemble spread and forecast skill (reliability)
- Use **8th order horizontal diffusion** for all resolutions
- Improved forecast skill and ensemble spread
- Introduce ESMF (Earth System Modeling Framework)
- Version 3.1.0r
- Allows concurrent generation of all ensemble members
- Needed for efficiency of stochastic perturbation scheme
- Add **new variables** (26 more) to pgrba files
- Based on NAEFS user request
- From current 52 to future 78 variables
- For NAEFS ensemble data exchange
- 								*Yuejian Zhu*

## GEFS 2009 IMPLEMENTATION TESTS

- Extend current 5-day skill to 6.5-day

*Yuejian Zhu, Dingchen Hou, Bo Cui, Dick Wobus*

- CRPS NH 850 hPa Temp

- RMSE / SPREAD 500 hPa Height

## GEFS IMPLEMENTATION PLANS FOR FY2010

**Real time generation of hind-casts**
- Make control forecast once every ~5th day
- T190L28, out to 16 days
- Use new reanalysis (~30 yrs)
- Why
- Increase sample of analysis – forecast pairs for statistical corrections
- Benefits
- Improved bias correction beyond 5 days
- Potential for regime / situation dependent bias correction
**Adaptive modification of initial and stochastic model perturbation variances**
- Based on recursive average monitoring of
- Forecast errors
- Ensemble spread
- Why
- Avoid having to tune perturbation size after each analysis/model/ensemble change
- Benefit
- Improved performance
- Easier maintenance
**Hydro-meteorological (river flow) ensemble forecasting**
- Pending on operational LDAS/GLDAS
**Coupled ocean-land-atmosphere ensemble**
- Couple MOM4 or HYCOM with land-atm component using ESMF
- Depending on skill, extend integration to 35 days
- Merge forecasts with CFS ensemble for seamless weather-climate interface
- Land perturbations (later)
- Why
- Explore predictability in intra-seasonal time scale
- Benefit
- Potential skill beyond 16 days
- Surface perturbations (4Q 2009)

## REAL-TIME GENERATION OF HIND-CAST DATASET

- Today’s Julian Date
- TJD

- TJD + 30

- TJD - 30

- Actual ensemble **generated today**

- Hind-casts for TJD+30 **generated today**

- Hind-casts (or its statistics) for TJD+/- 30 **saved on disc**

- Time

- 2005

- 2004

- 2003

- 1968

- 1967

**REAL-TIME GENERATION OF HIND-CAST DATASET**

- 2006

***BACK***

## SREF DEVELOPMENT PLAN

**Stochastic physics**
- Convective parameterization of Teixeira et al - NRL
- Stochastic feedback from Grell-Devenyi
**Ensemble Transform** initial perturbations
- Consistent with boundary perturbations from GEFS
- River flow, wave, etc **ensemble applications**
- Explore use in targeting observations, too
**Resolution**
- Next upgrade ~ 20km
**Infrastructure**
- Model diversity under NEMS
- 								*Jun Du*

## HIGH IMPACT ENSEMBLE SYSTEM

**Motivation**
- Support probabilistic high impact event forecasting
**Potential applications**
- Severe / fire / aviation weather, AQ, tropical / winter storm, etc
**General framework**
- Relocatable / movable mesh embedded into SREF
- Model diversity etc configured differently for each application
-  **Deployment**
- On demand, based on user request / priority list of the day
- Like Winter Storm Reconnaissance requests for observations
**Operational execution**
- High impact “window” = hurricane window?
**Potential partners**
- HWTB, RR, HFIP, ensemble-DA, etc
**Status**
- Initial planning phase
- Mesh capability being worked on
- Possible testing from 2010?

##  grid221 (30km)

-  grid221 (30km)

## ENSEMBLE DATA PROCESSING STEPS 

**Bias correction / combination of all info**
- Remove lead time dependent model bias on model grid
- Calibrate higher moments of ensemble
- Combine information from all sources into single set of ensemble
**Forecaster modification**
- Subjective changes to ensemble data (over US only?)
**Proxy for truth**
- Create observationally based fine resolution analysis for use in downscaling
**Downscaling**
- Interpret bias corrected ensemble on user relevant grid – NDFD
**Additional variables**
- Derive further variables from bias corrected / downscaled NWP output
**Visualization / Derived products**
- Interrogate bias corrected / downscaled ensemble dataset

## BIAS CORRECTION OF FORECASTS

**Method**
- Bayesian processor
- Combines information from all sources
- Fuses forecast data with climatological distribution (“prior”)
- Adjusts “spread” according to skill observed in forecast sample
- Outputs statistically corrected distribution (“posterior”)
- Ensemble members adjusted to represent posterior distribution
**Data sources**
- Reanalysis as prior (use new reanalysis when available)
- Sample of past forecasts - most recent 60-90 days
- Include control hind-casts when available
- Latest analyzed or observed data
**Current status**
- 35 NAEFS & SREF variables bias corrected
- 1st moment corrected only
- NAEFS & SREF processed separately
- CMC + NCEP ensembles + GFS hires control combined
**Plan**
- Bias correct all model output variables on model native grid (~200, 2-3 yrs)
- Include precipitation (use observationally based analysis as truth – 1-2 yrs)
- Add hind-casts for NCEP ensemble control (1-2 yrs)
- Process ensembles & hires forecasts from FNMOC, ECMWF (1-2 yrs)
- Combine all forecasts into single guidance (2-3 yrs)
- Ensemble & hires from NCEP, CMC, FNMOC, ECMWF, and SREF

## List of Variables for Bias Correction, Weightsand Forecast Anomalies for CMC & NCEP Ensemble

*Yuejian Zhu*

## Bo Cui

- 		BEFORE 			         AFTER
- BIAS CORRECTION

*Bo Cui*

##   IMPACT OF BIAS CORRECTION ON

-   **IMPACT OF BIAS CORRECTION ON**
- ESTIMATED SYSTEAMTIC ERROR     PROBABILISTIC SCORES

- NH 500hPa height

- _850hPa temperature_

*Before bias correction (**1*x*1**)*

*After bias correction (1*x*1)*

**NH 2m temperature**

- Before Bias Correction

- After Bias Correction

- NH 500hPa height

- Tropics 500hPa height

*Bo Cui*

## PROXY FOR TRUTH

**Method**
- Observationally based data assimilation
- If NWP first guess is used
- Consider bias correcting first guess
- Do not cycle first guess from observationally based analysis
- Allows to draw closer to data
**Data sources**
- Near surface observations
- NWP first guess (if good quality available)
**Current status**
- Real Time Meso-scale Analysis (RTMA)
- On NDFD grid
- RUC as first guess
- 2d version of GSI as DA procedure
- Fields
- 2m temp, 10m wind (u, v, speed, direction), sp
- Domains
- CONUS, Alaska (Hawaii, Puerto Rico, Guam coming)
- Combined RFC – CPC CONUS precipitation analysis
- On NDFD grid
**Plan**
- Include more variables
- Improve DA method
- Incorporate bias correction of FG
- Extend precip to global coverage with use of NWP FG & RMORPH data

## Mike Charles

- RFC 4X4 km grid

- CPC 1/8 grid

- Combined, 5x5 km grid

*Mike Charles*

## DOWNSCALING

**Method**
- Perfect prog
- Establish relationship (“***donwscaling vector***”) between
- Proxy for truth (high resolution observationally based analysis) &
- NWP analysis (used as reference in bias correction step)
- Level of sophistication
- Climatological (statitical)
- Regime dependent (statistical)
- Case dependent (dynamical, using LAM) – most expensive
- Sub-NWP-grid resolution variance
- Need to be stochastically added in statistical methods
- Outputs ensemble members statistically consistent with
- Bias corrected forecasts on NWP grid
- Proxy for truth on fine resolution grid
**Data sources**
- Sample of
- Fine resolution observationally based analysis fields
- Corresponding NWP analysis fields
**Current status**
- 4 NDFD variables downscaled using regime dependent downscaling vector
- 2m temp, 10m u&v, surface pressure
- RTMA used as proxy for truth
**Plan**
- Add more NDFD variables by
- Expanding RTMA analysis & using derived variables (eg, wind speed / direction)
- Using SMARTINIT + downscaled NDFD variables

## MDL GMOS  & NAEFS Downscaled Forecast  

- 24-h GMOS
- Forecast

- For CONUS:
- NAEFS(1.45) : GMOS(1.72)
- 19% impr. over GMOS

- MDL GMOS  & NAEFS Downscaled Forecast
- Mean Absolute Error w.r.t. RTMA  Average For Sept. 2007

- 24-h NAEFS Forecast

*Valery Dagostaro, Kathy Gilbert,*
*Bo Cui, Yuejian Zhu*

## Bo Cui, Yuejian Zhu

- _GMOS forecast_

- _NAEFS products_

- Surface Temperature MAE
- CONUS, Sept. 2007
-  12Z NDFD vs. 00Z MOS/_GMOS_/_NAEFS_

- Surface Temperature MAE
- CONUS, Sept. 2007
-  00Z _GMOS_ vs. 00Z _NAEFS _

- RTMA Analysis

- METAR  obs. 1221 sites

**0.5°C**

**0.5°C**

*Bo Cui, Yuejian Zhu*

*Valery Dagostaro, Kathy Gilbert*

## Bo Cui, Yuejian Zhu

- CONUS, Sept. 2007
-  12Z NDFD vs. 00Z MOS/_GMOS_/_NAEFS_

- Surface Temperature Pointwise Bias
- CONUS, Sept. 2007
-  00Z _GMOS_ vs. 00Z _NAEFS_

- _GMOS forecast_

- _NAEFS products_

- RTMA Analysis

- METAR  obs. 1221 sites

- Pointwise Bias

- Area Mean Bias

**0.6°C**

**0.6°C**

*Bo Cui, Yuejian Zhu*

*Valery Dagostaro, Kathy Gilbert*

## DERIVED VARIABLES 

**Objective**
- Generate variables not carried in NWP models
**Input data**
- Bias corrected and downscaled ensemble data (NWP model output)
**Methods**
- Model “post-processing” algorithms
- Apply after downscaling for variables affected by surface processes
- SMARTINIT
- NDFD weather element generator
- Other tools?
- Text generation, etc?

*Geoff Manikin et al.*

## STATISTICAL PROCESSING OF PRECIPITATION 

**Problem**
- Precipitation is discontinuous
- Traditional approach to calibrated PQPF
- Devise method to calibrate POP
- Devise method to calibrate conditional QPF
**Solution**
- Define new continuous variable – pseudo precipitation
- Equals precipitation when p>0
- Proportional to moisture deficit in vertical column
- Use generic Bayesian ensemble processor
**Status**
- Generic Bayesian ensemble processor – in collab. w R. Krzysztofowicz
- Algorithm complete - in collaboration with
- Significant part of codes transferred from Univ. VA
- Next - Assemble and test codes
- Pseudo precipitation data – in collaboration with Paul Schultz
- Combined CPC & RFC observed precip dataset complete – in collab. w. OHD
- Pseudo precip variable defined, dataset being produced
- Next – Prepare climatological “prior” for pseudo precip
**Plan**
- Test / implement ensemble processor for 6-hr precip in 2009 / 2010
- Adjusted/calibrated ensemble members / PQPF

## Application of Pseudo-precipitation

- With a single continuous distribution, can bias correct in *one step*
- Shift entire ensemble distribution to remove bias in PP
- For example, if some members never precipitate in an area with frequent drizzle...Bias correction shifts the PP distribution to the right, forcing some of those members into the positive PP (forcing them to precipitate)

*Paul Schultz, Mike Charles*

## END-TO-END FORECAST PROCESS

- modeling

**END-TO-END FORECAST PROCESS**

-  **OBSERVATIONS**

-  **USERS**

**USERS**
- Applications

- NCEP, other centers
- Ensemble generation

- MDL/NCEP, partners
- Ensemble processing

- NCEP/SS **** WFOs
- QC, enhancements

- Statistical
- calibration

- Human
- intervention

***HOW***

***WHO / WHAT***

***WHY***

- Dynamical
- Prediction

- Statistical
- consistency

- Added value
- Interpretation

- Decision
- models

## BACKGROUND

## USER REQUIREMENTS

**Decisions regarding operations affected by weather**
- Must consider information on weather to
- Minimize losses due to adverse weather
- Maximize benefits from favorable weather
- Different applications affected by
*Various weather parameters*
**Optimal user decision threshold** equals
- Probability of adverse weather exceeding
- Cost / loss ratio of decision situation (simplified decision theory)
- Decisions have their *unique cost / loss environment*
*Joint probabilities* needed when multiple weather parameters used
***Decision maker must know probability of adverse weather***
**Probability of weather events** can be derived from ensemble
- Probabilities for all variables affecting users, incl. joint probabilities
**Ensemble dataset as primary media/carrier of forecast** info
- All relevant variables must be included
- Suitable number of ensemble members for joint probabilities

## RTMA EVALUATION

- Experimental setup
- Cross-validation against data withheld from analysis
- Average for seven cases for May 3 2007, 12Z - 18Z
- RMS difference between independent observations and
- 1-hr RUC forecast (First Guess)
- RTMA analysis
- Benchamrk – Cressman analysis with one pass
- Results
- 10m wind (m/s)
- Guess 		RTMA 		Baseline (Cressman)
- 2.96 		1.43		2.41
- 2m temperature (K)
- Guess 		RTMA 		Baseline (Cressman)
- 2.20 		1.26 		1.73
- 2m specific humidity (g/kg)
- Guess 		RTMA 		Baseline (Cressman)
- 0.10E-02 	0.49E-03 		0.80E-03
- Conclusion
- Combining information from numerical FG & observations benefits RTMA
- ~50% better fit to data than numerical FG only
- 30-40% better fit than statistical benchmark analysis based on observations only

*Manuel Pondeca*

## STATISTICS

-                     (all obs)       (all obs)         (cval)        (cval)

**STATISTICS**

**RTMA EVALUATION - 2**

- Note: Results represent 10-day averages from 26 Oct through 4 Nov 2008. Columns 2 and 3 show results from the original gsi runs, ie., they display statistics computed over the entire set of obs used in the analysis.  Columns 4 and 5 display results averaged over 5 independent cross-validation datasets. Each cross-validation set contains roughly 10% of all the data

*Manuel Pondeca*

## CONNECTING DECISION MAKING WITH FORECAST INFO

- Option 1 - Weather forecaster prepares “probabilistic products” for users
- Eg, probability of significant convection in vicinity of 2 specific hubs between 3-6pm
- Only option in past, based on error statistics of single value forecasts
- Problems
- Ever growing list of special products needs to be generated
- Still not covering all situations – Decision making compromised
- Estimate cost/loss ratio associated w. decision regarding adverse weather event
- User acts if probability of adverse event higher than cost/loss ratio
- Limitations
- Assumes prior analysis & knowledge of
- Weather sensitive operations & associated cost/loss ratio
- Sub-optimal decisions in cases deviating from analyzed situations
- Option 2 - Weather forecaster presents user with multiple scenarios
- Ensemble weather data in NEXGEN
- Space (3D) plus time plus variables plus ensemble =  ***6D-Cube***
- Develop general model simulating user operations including costs
- Optimize weather related operations based on climatology
- Vary procedures in operations that are sensitive to weather
- For each set of procedures, simulate operations using all ensemble weather scenarios
- Take ensemble mean of economic outcome (costs + losses) for each set of procedures
- Choose operating procedures minimizing costs + losses in expected (ensemble mean) sense
- Advantages
- No prior knowledge required except general model of operations/costs
- General approach to optimizing weather related decisions

## PARTNERS / OUTREACH 

- Multi-center ensemble combination
- NAEFS - Env. Canada; TIGGE – GIFS (WMO/THORPEX) – Other ensemble NWP centers/users
- Statistical processing – Bayesian processor for ensembles
- Roman Krzysztofowicz, Univ. VA
- RTMA, Smartinit
- WFOs
- Hindcasting
- Tom Hamill, PSD/ESRL
- Pseudo precipitation, downscaling
- Paul Schultz, GSD/ESRL
- Precipitation analysis
- PingPing Xie, CPC/NCEP, DJ Seo, OHD
- Probabilistic verification
- Julie Demargne & James Brown, OHD, NPVU
- Intercomparison of post-processing results
- Kathy Gilbert, MDL
- Applications
- Hydromet – OHD, HPC
- Climate - CPC
- etc

## WORK ARRANGEMENTS 

*Develop environment conducive of joint work*
- Communication among partners critical
- NFUSE team on ensemble / probabilistic processing
- Encourage complimentary efforts
- Modular design
- Enable inter-comparison of competing methods
- Common/shared
- Verification tools
- Datasets
- What is role of OSIP?
- Suited to guide innovative development work?

## EMC-MDL COLLABORATION

- Compare quality of current operational / experimental products
- Gridded MOS vs. Downscaled NAEFS
- Ongoing
- Kathy Gilbert, Val Dragostano – Zoltan Toth, Bo Cui, Yuejian Zhu
- Proxy for truth issue unresolved
- Need observations independent of MOS
- MDL experimental ensemble guidance vs. Downscaled NAEFS
- 10/50/90 percentiles to be evaluated
- Matt Peroutka & Zoltan Toth
- Proxy for truth issue
- Proxy for truth?
- Agree on best proxy for truth
- Collaborate on
- Improving RTMA, including bias correction for FG
- Creating best CONUS precipitation analysis & archive
- Joint research into best downscaling methods?
- Climate, regime, case dependent methods
- Addition of fine temporal/spatial variability into ensemble

## KEY POINTS

**6D-Cube** - Dataset of bias corrected/calibrated ensemble members
- Variable, space (3D), time,  ensemble member
- All forecast information can be derived
**Mass production** of bias corrected ensemble dataset
- Currently 35 variables, 200+ planned
- Very **strong external links** to accelerate progress
- NAEFS, NUOPC, THORPEX – TIGGE / GIFS, etc
**Methods** for processing must be
- General & versatile for mass production
- Modular for collaboration
**Transition to operations** – unique job
- Select best methods for each module in processing
- Test / implement in stream of jobs
**Seamless links** needed for use in end-to-end process
- NWS concept of operations, NFUSE

## BACKGROUND

## CONNECTING DECISION MAKING WITH FORECAST INFO

**TRADITIONAL APPROACH**
- Provide “probabilistic products” for users
- Only option in past, based on error statistics of single value forecasts (eg, MOS POP)
- Eg, probability of significant convection in vicinity of 2 specific hubs between 3-6pm
- Now can be based on ensemble
- Users act when forecast probability exceeds their cost/loss ratio
- Advantage
- Can derive probabilities from single forecast
- Problems
- Proliferation of number of products
- Different variables, probability/weather element thresholds, joint probabilities
- Limited usage
- Downstream applications severely limited (eg, wave, streamflow, etc, ensembles not possible)
**PROPOSED APPROACH**
- Provide multiple (N) plausible weather scenarios
- Ensemble weather data
***6D-Cube*** - space (3D) + time + variables + ensemble
- Expanded NDFD – future official NWS weather / climate / water forecast database
- Develop user application model simulating optimal weather related operations including
- Actions / costs / benefits
- Run application model NxN times with
- Weather scenario from each ensemble member using
- User procedures optimized for each ensemble member
- Take ensemble mean of economic outcome (costs + losses) for each set of user procedures
- Choose operating procedures minimizing costs + losses in expected (ensemble mean) sense
- Advantages
- General approach to optimizing weather related decisions
- Focus on optimization of
- Forecast ensemble (including statistical reliability)
- User application model

## CONNECTING DECISION MAKING WITH FORECAST INFO

**TRADITIONAL APPROACH**
- Provide “probabilistic products” for users
- Only option in past, based on error statistics of single value forecasts (eg, MOS POP)
- Eg, probability of significant convection in vicinity of 2 specific hubs between 3-6pm
- Now can be based on ensemble
- Users act when forecast probability exceeds their cost/loss ratio
- Problems
- Proliferation of number of products
- Different variables, probability/weather element thresholds, joint probabilities
- Limited usage
- Downstream applications severely limited (eg, wave, streamflow, etc, ensembles not possible)
**PROPOSED APPROACH**
- Provide multiple (N) plausible weather scenarios
- Ensemble weather data
***6D-Cube*** - space (3D) + time + variables + ensemble
- Expanded NDFD – future official NWS weather / climate / water forecast database
- Develop user application model simulating optimal weather related operations including
- Actions / costs / benefits
- Run application model NxN times with
- Weather scenario from each ensemble member using
- User procedures optimized for each ensemble member
- Take ensemble mean of economic outcome (costs + losses) for each set of user procedures
- Choose operating procedures minimizing costs + losses in expected (ensemble mean) sense
- Advantages
- General approach to optimizing weather related decisions
- Focus on optimization of
- Forecast ensemble (including statistical reliability)
- User application model

## CONNECTING DECISION MAKING WITH FORECAST INFO

**TRADITIONAL APPROACH**
- Provide “probabilistic products” for users
- Only option in past, based on error statistics of single value forecasts (eg, MOS POP)
- Eg, probability of significant convection in vicinity of 2 specific hubs between 3-6pm
- Now can be based on ensemble
- Users act when forecast probability exceeds their cost/loss ratio
- Problems
- Proliferation of number of products
- Different variables, probability/weather element thresholds, joint probabilities
- Limited usage
- Downstream applications severely limited (eg, wave, streamflow, etc, ensembles not possible)
**PROPOSED APPROACH**
- Provide multiple (N) plausible weather scenarios
- Ensemble weather data
***6D-Cube*** - space (3D) + time + variables + ensemble
- Expanded NDFD – future official NWS weather / climate / water forecast database
- Develop user application model simulating optimal weather related operations including
- Actions / costs / benefits
- Run application model NxN times with
- Weather scenario from each ensemble member using
- User procedures optimized for each ensemble member
- Take ensemble mean of economic outcome (costs + losses) for each set of user procedures
- Choose operating procedures minimizing costs + losses in expected (ensemble mean) sense
- Advantages
- General approach to optimizing weather related decisions
- Focus on optimization of
- Forecast ensemble (including statistical reliability)
- User application model

## PROPOSED APPROACH

**SHORT TERM** (2-3 years) – Traditional approach as stopgap solution
- Users
- Identify key areas of weather sensitive operations
- Define limited set of associated probabilistic products
- Estimate related cost/loss ratios
- Forecasters
*Produce limited set of probabilistic products based on ensemble*
- Users vary their action depending on
- Forecast probability AND
- Cost/loss ratio
**LONG TERM** (5-10 years) – New approach
- Forecasters
- Provide easy access to high quality ensemble data – NEXGEN 6D-Cube
- Statistically calibrate raw ensemble data
- Expand current NAEFS / MOS capabilities
*Make accessible all statistically calibrated ensemble data*
- Expand current TOC dataset
*Provide smart sub-setting & interrogation tools*
- Expand functionalities of current NOMADS technology
- Users
- Develop models simulating their operations
- Develop techniques to optimize weather related decisions based on ensemble expectation

*NFUSE - Doug Hildebrand*

## SREF DEVELOPMENT PLAN

- Expand bias correction from only CONUS domain (grid212) to Alaska and Hawaii two grids (grids 243 and 216) or to bigger North American domain (grid221)
- Dynamically downscaling the 32km SREF using 12km NAM forecasts (dual-resolution ensemble approach)
- Statistically downscaling the 32km SREF using 5km RTMA
- Bias correcting SREF precipitation fields
- NEMS-only based SREF: 20km, stochastic physics as well as regional ET (in a later time)
- Explore High-Impact Weather Ensemble Forecast system (HWEF)

## NAEFS STATISTICAL CORRECTION / PRODUCT PLANS FOR FY2009

- Downscaling
- CONUS
- Additional variables (Tmax and Tmin, wind speed and direction)
- Pending on RTMA availability
- Alaska
- 8 variables (T2m, Tmax, Tmin, Psfc, U10m, V10m, 10 meter Ws and Wd)
- In testing
- In collaboration with HPC Alaska Desk
- Other regions
- Hawaii, Puerto Rico and Guam
- Pending on RTMA availability
- New NAEFS variables for data exchange
- 25 additional variables
- Use GRIB2 format – faster transfer (30 min saving)
- Coordinated with CMC/MSC
- Dedicated line for NCEP and CMC NAEFS data exchange
- DS-3?
- Earlier access to NAEFS ensemble & products

## NAEFS final products

- From 	Bias correction (NCEP, CMC)
- 	Dual-resolution (NCEP only)
- 	Down-scaling (NCEP, CMC)
- 	Combination of NCEP and CMC

- _NAEFS final products_

- _NCEP/GEFS raw forecast_

- 8+ days gain

## NAEFS STATISTICAL CORRECTION / PRODUCT PLANS FOR FY2010

- Bias correction
- Develop, test, implement Bayesian bias correction technique
- Apply on all prognostic model variables
- Including “pseudo-precipitation”
- Downscaling
- Improve methods
- Apply to new variables

## NAEFS EXPANSION PLANS FY2009/10

- Incorporate other ensemble data into NAEFS
- One year evaluation period
- Must coordinate with CMC
- Use “mini-Bayesian” for testing
- FNMOC global ensemble
- Need to resolve the problem for data transform (still missing forecast data for FNMOC)
- ECMWF global ensemble
- Need to coordinate with NCO for 00UTC data and 6hr forecast intervals
- Probabilistic verification
- Unified probabilistic verification (3Q, 2009)
- Shared codes with SREF
- Partnership with OHD, etc

## NAEFS future configuration

|  | NCEP | CMC |
| --- | --- | --- |
| Model | GFS | GEM |
| Initial uncertainty | ETR | EnKF |
| Model uncertainty | None | Yes |
| Stochastic physics | Yes | Yes  |
| Tropical storm | Relocation | None |
| Daily frequency | 00,06,12 and 18UTC | 00 and 12UTC |
| Resolution | T190L28 (d0-d16)~70km | (d0-d16) ~1.0degree |
| Control  | Yes | Yes |
| Ensemble members | 20 for each cycle | 20 for each cycle |
| Forecast length | 16 days (384 hours) | 16 days (384 hours) |
| Post-process | Bias correction
for ensemble mean | Bias correction 
for each member |
| Last implementation  | December 2008 (plan) | July 10th 2007 |

_**NAEFS future configuration**_
- _Updated: October 2008_

## CURRENT CONFIGURATION

**NORTH AMERICAN ENSEMBLE FORECAST SYSTEM (NAEFS)**
- Definition
- Multi-center ensemble system combining
- 20 perturbed plus a control member out to 16 days twice a day on 1x1 lat/lon grid from
- NCEP - Global Ensemble Forecast System (four times per day)
- T126L28, ET initial perturbation technique, no model perturbations
- Canadian Meteorological Center (CMC)
- Operational steps
- Generation NCEP global ensemble
- Real time data exchange with CMC (50+ variables)
- Bias correction of both ensembles (35 variables)
- Incorporation of information from high resolution GFS
- Downscaling of 1x1 bias corrected fields onto 5x5 km NDFD grid (4 variables)
- Product generation – see next page

## PRODUCTS

**NORTH AMERICAN ENSEMBLE FORECAST SYSTEM (NAEFS)**
- Basic products on 1x1 lat/lon grid
- Bias corrected ensemble forecasts (42 members, 35 variables)
- Climate anomaly forecasts for downscaling applications (42 members, 19 variables)
- Derived products
- Mean, mode, 10, 50 (median), 90 percentile forecasts, spread on 1x1 lat/lon grid
- Downscaled forecasts on 5x5 km grid (4 variables)
- Week-2 temperature forecasts (jointly with CPC)
- GEMPAK grids / images
- Ensemble / probabilistic displays
- Distribution of products
- Mainframe / NAWIPS for NCEP Service Centers
- AWIPS for WFOs
- WOC & NCEP ftp servers for grids for external users
- Web sites for  images
- NCEP official web site
- EMC experimental web site

## CUSTOMERS

**NORTH AMERICAN ENSEMBLE FORECAST SYSTEM (NAEFS)**
- NCEP Service Centers
- Gridded fields, NAWIPS manipulation tools, images
- NCEP International Desks
- South American Desk
- African Desk
- WFOs
- AWIPS grids, ftp’d grids, images
- RFCs
- Grids (experimental)
- External users
- Ensemble grids, images
- NAEFS partners
- CMC
- National Meteorological Service of Mexico (NMSM) – images
- International community
- Severe Weather Forecast Demonstration Projects
- Private companies

## NWS Seamless Suite of ForecastProducts Spanning Climate and Weather

- Linkage

***Forecast ***
***Uncertainty***

***Minutes***

***Hours***

***Days***

***1 Week***

***2 Week***

***Months***

***Seasons***

***Years***

- NWS Seamless Suite of ForecastProducts Spanning Climate and Weather

- North American Ensemble
- Forecast System

- Climate Forecast System*

- Short-Range Ensemble Forecast

- Ocean Model
- Hurricane Models

- Global Forecast System

- North American Forecast

- Rapid Update Cycle for Aviation

- Dispersion Models for DHS

- GFDL
- WRF

**NCEP Model Perspective**

- Maritime

- Life & Property

- Space Operations

- Recreation

- Ecosystem

- Environment

- Emergency Mgmt

- Agriculture

- Reservoir Control

- Energy Planning

- Commerce

- Hydropower

- Fire Weather

- Health

- Aviation

## Slide 52

## D5=0.357

- _D5=0.357_
_**2001**_

- _D6=0.334_
_**2007**_

- _D5=0.356_

- _D6=0.361_

## NCEP/GEFS raw forecast

- _NCEP/GEFS raw forecast_

- _NAEFS final products_

- 4+ days gain from NAEFS

- From 	Bias correction (NCEP, CMC)
- 	Dual-resolution (NCEP only)
- 	Down-scaling (NCEP, CMC)
- 	Combination of NCEP and CMC

## Latest retrospective run (full package)

- _NH 500hPa height_
- _RMSE & Spread_

- _SH 500hPa height_
- _CRPSS_

- _NH 2-m temperature_
- _RMSE & Spread_

- _NH 500hPa height_
- _CRPSS_

- _E20s – T126L28_
- _E20g – T190L28 (0-180 only)_

_**Latest retrospective run (full package)**_

## Slide 56

## NH Anomaly Correlation for 500hPa HeightPeriod: August 1st – September 30th 2007

- GEFSg is better than GFS at 48 hours

- GEFSg could extend skillful forecast (60%) for 9+ days
- 24 hours better than current GEFS
- 48 hours better than current GFS

## Plans for FY2010 

- GEFS
- Configuration:
- Variable resolutions:
- T270L42 (0-180hr) (considering half-degree products)
- T190L28 (180-384hr)
- T126L28 (384-840hr)
- Full coupling with ocean model (assume in)
- Science:
- Improving TS relocation
- Adopt all new developed TS relocation schemes
- Improving stochastic scheme
- Adaptive 2/3-demisional parameters adjustment

## Plans for FY2010 (Cont.)

- NAEFS
- New NAEFS component – FNMOC global ensemble
- 4Q 2009 – 2Q 2010
- Pending on one year evaluation (May 2008 – April 2009)
- Using mini-Bayesian method for first moment correction
- Need to coordinate with CMC/MSC
- Improving NAEFS products
- Introduce full Bayesian model to calibrate high moments
- For precipitation forecast
- All variables
- Statistical down-scaling
- Precipitation
- Improving current method
- Adding new variables (pending on RTMA availability)
- TC related products
- Including bias correction
- Seamless weather-climate interface
- Merge GEFS and CFS

## Background!!!

## Acknowledgements

- EMC: Richard Wobus, Dingchen Hou, Bo Cui
- 	Malaquias Pena, Weiyu Yang, Julia Zhu,
- 	Mozheng Wei, Mike Charles,
- 	Yucheng Song, Jun Du, Mark Iredell,
- 	John Ward and Steve Lord
- CPC: Qing Zhang, Jon Gottschalck, Jae Schemm
- NCO: Christine Caruso Magee, Joey Carr, Brent Gorden,
- 	Daniel Starosta
- MDL: Kathryn Gilbert
- MSC/Canada: Lewis Poulin and Andre Methot

## CCS resources (estimated)

- Computation (Current)
- T126L28 out to 384 hours
- Assigned window (75min)
- Actually using 45 minutes
- Average 38 nodes
- Computation (future)
- T190L28 out to 384 hours
- Use 50 min
- Average 60 nodes
- 75% additional computer resources

- Space (current)
- T126L28 out to 384 hours
- Pgrba files
- 17 days on CCS for bias correction
- 55G (x4 per a day)
- Space (future)
- T190L28 out to 384 hours
- Pgrba files
- 17 days on CCS for bias correction
- 83G needed (x4 for a day)

## NEXT NAEFS exchange pgrba files

|  Variables | Pgrba file | Total 78 (26) |
| --- | --- | --- |
| GHT | Surface, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa | 11 (3) |
| TMP | 2m, 2mMax, 2mMin, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa | 13 (3) |
| RH | 2m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa | 11 (3) |
| UGRD | 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa | 11 (3) |
| VGRD | 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa | 11 (3) |
| PRES | Surface, PRMSL | 2 (0) |
| PRCP (types) | APCP, CRAIN, CSNOW, CFRZR, CICEP | 5 (0) |
| FLUX (surface) | LHTFL, SHTFL, DSWRF, DLWRF, USWRF, ULWRF | 6 (6) |
| FLUX (top) | ULWRF (OLR) | 1 (1) |
| PWAT | Total precipitable water at atmospheric column | 1 (0) |
| TCDC | Total cloud cover at atmospheric column | 1 (0) |
| CAPE | Convective available potential energy | 1 (0) |
| SOIL | SOILW(0-10cm), WEASD(water equiv. of accum. snow depth), SNOD(surface), TMP(0-10cm down) | 4 (4) |
|  |  | 26 new vars |
| Notes | Surface GHT is only in analysis file and first pgrb file when the resolution changed.
23 of 26 new variables are from pgrbb files, 10, 50hPa RH and SNOD are new variables |  |

- _NEXT NAEFS exchange pgrba files_

## NEXT NAEFS pgrba_bc files

|  Variables | pgrba_bc file | Total 48 (13) |
| --- | --- | --- |
| GHT | 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa | 10 (3) |
| TMP | 2m, 2mMax, 2mMin, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa | 13 (3) |
| UGRD | 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa | 11 (3) |
| PRES | Surface, PRMSL | 2 (0) |
| VGRD | 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa | 11 (3) |
| FLUX (top) | ULWRF (toa - OLR) | 1 (1) |
|  |  | 13 new vars |
| Notes |  |  |

- _NEXT NAEFS pgrba_bc files_
- _(bias correction)_

## Horizontal resolution change

- _26 : 4_

_**Horizontal resolution change**_
- Ensemble control only (deterministic)
- From T126 to T190
- NH 500hPa geopotential height

- _Gains from short waves_

## May 2007

- OPR(T126)-4th order
- NHD(T126)-8th order

- _May 2007_

- _November 2007_

_**Horizontal diffusions**_
- Ensemble controls only

## Resolution and Diffusion for Global Ensemble Without Stochastic

- E20s – T126 4th for all 16d (oper.)
- E20x – T190 8th out to 16d
- E20e – T190 8th (0-180h), then T126 4th

- When reducing resolution from T190 (8th order) to T126 (4th order), the ensemble forecast probabilistic skill score tends to t126 immediately, the example shows here for tropical 850hPa temperature. 8th order diffusion for t126 somewhat improves performance (not show here). Therefore, both the resolution and diffusion play an important role here.

_**Resolution and Diffusion for Global Ensemble Without Stochastic**_

## Conclusion

- Based on two sets of retrospective runs (summer and winter 2007)
- New package improved the forecast skill (score) significantly
- For deterministic (ensemble mean)
- For probabilistic (ensemble distribution)
- The better results is benefited from:
- Increase horizontal resolution (include diffusion)
- Stochastic perturbation scheme
- Better initial condition (analysis)
- Better forecast model (GFS)

## Remain Issues

- Tune initial perturbation (may leave this to next implementation)
- Need to adjust the size of initial perturbation, due to:
- Model resolution changed
- Model diffusion scheme changed
- Improved analysis
- Experiments are running, but very slow
- Due to limit computation resource
- Examine bias corrected forecast and down scaling forecast
- No enough CCS disk storage for 17d pgrb files on line
- Verify tropical storm tracks
- Working on 2007 summer season
- Planned for 2008 summer season
- Resource problem
- Personnel
- Computation and storage

## Downstream Dependencies

- Sigma files
- SREF
- Yes
- It uses sigma forecast
- Wave ensemble
- No
- It uses bias corrected 10m winds
- Tracking
- No
- It uses pgrba file
- MDL GMOS
- No
- It uses pgrba and pgrbb files
- Public access
- No
- We don’t post sigma files to public

- pgrb files (pgrba + pgrbb)
- SREF
- No
- It produces pgrb file by itself
- Wave ensemble
- Yes
- But file has the same format for 10m wind
- Tracking
- Yes
- But it uses pgrba file only, the file has the same format
- MDL GMOS
- Yes
- It uses both pgrba and pgrbb
- Public access
- Yes
- pgrba and pgrbb