# Presentation: 111866

## May 2006 Upgrade of the GEFS and First Implementation of NAEFS systems

- Yuejian Zhu,
-  Zoltan Toth, Richard Wobus, Mozheng Wei and Bo Cui
- Environmental Modeling Center
- NOAA/NWS/NCEP
- Acknowledgements:
- DingChen Hou, Ken Campana, Mark Iredell and Stephen Lord  EMC
- David Michaud, Brent Gordon and Luke Lin  NCO

## NAEFS Background Information

- First of a kind project
- Operational multi-center ensemble system
- Bias correction, climate percentiles never computed on such a scale operationally
- Timetable
- Mar 2003	Project started
- Oct 2003	Draft Research, Development and Implementation Plan
- Sep 2004	Initial Operational Capability – Operational data exchange
- May 2006	First Operational Implementation
- Challenges
- Developed joint plan with MSC personnel
- Arranged operational data exchange
- Coordinated GEFS development with international NAEFS developments
- Coordinated software development & operational implementation with MSC
- Worked with less THORPEX resources than planned originally
- Future expansion
- Develop sustainable plans
- Coordinate with partners
- Rename NAEFS and position it as prototype GIFS system
- Resource concerns
- Computational (telecommunication, disc, etc)
- Human

## Planned Changes for GEFS- Summary

- Increase the number of perturbed ensemble members
- 14 (in place of current 10) perturbed runs for each cycle (20 by early 2007)
- NAEFS requirement
- This change is intended to improve ensemble based prob. forecasts
- Results: improved probabilistic skill, slightly improved ensemble mean skill (seasonally dependent)
- Add control runs for 06, 12 and 18Z cycles
- This change is intended to enable for relocation of perturbed tropical storm
- Facilitates comparison of high & lower resolution ensemble controls
- If lores control and ensemble mean differ – indication of nonlinearities
- If high & lores controls differ – indication for possible effect of resolution
- Introduce Ensemble Transform (ET) into GEFS breeding method
- ET breeding method creates globally orthogonal initial perturbations
- Uses simplex method to create individual (not paired) perturbations
- This change is intended to improve probabilistic forecast skill
- Results: Improved probabilistic forecast skill; Slightly reduced ensemble mean hurricane track errors for 12-96 hours
- Changes of File Names and Structures

## GEFS configurations

|  | Current | New |
| --- | --- | --- |
| Model | GFS | GFS |
| Initial uncertainty | BV | ETBV |
| Model uncertainty | None | None |
| Tropical storm | Relocation | same |
| Daily frequency | 00,06,12 and 18UTC | same |
| Hi-re control 
(GFS) | T382L64 (d0-d7.5)
T190L64 (d7.5-d16) | same |
| Low-re control (ensemble control) | T126L28 (d0-d16)
00UTC only | T126L28 (d0-d16)
00,06,12 and 18UTC |
| Perturbed members | 10 for each cycle | 14 (20) for each cycle |
| Forecast length | 16 days (384 hours) | same |
| Implementation  | August 17th 2005 | May 30th 2006 |

_**GEFS configurations**_

## Ensemble Transform Bred Vector

- P1

- N1

- P2

- N2

- P#, N# are the pairs of positive and negative
- P1 and P2 are independent vectors
- Simple scaling down (no direction change)

- _Ensemble Transform Bred Vector_
- (New)

- P1, P2, P3, P4 are orthogonal vectors
- No pairs any more
- To centralize all perturbed vectors (sum of all vectors are equal to zero)
- Scaling down by applying mask,
- The direction of vectors will be tuned by ET.

- _Rescaling_

**ANL**

**ANL**

**ANL**

- _Bred Vector_
- (Current)

_**P1 forecast**_

_**P4 forecast**_

_**P3 forecast**_

_**P2 forecast**_

- t=t1

- t=t0

- t=t0

- t=t2

- t=t2

- t=t1

- _Rescaling_

## Changes of File Names and Structures

- Pressure GRIB Files Split into Two
- Pgrba – 51 Variables For NAEFS Exchange
- Pgrbb – Remaining 278 Variables
- Perturbation runs from pairs to single size
- P1, n1, p2, n2 ... convert to p0, p02, p03, p04 ...
- Enspost.* and Ensemble.* Files Eliminated
- Data Was Duplicate to Pressure GRIB Data Packed in Different Format
- Ensemble Extensions Corrected in Pressure GRIB Files
- 6-Hourly Precipitation/Max/Min Accumulations Available in Pressure GRIB Files
- GEMPAK Files Created for NAEFS Members for Raw and Bias Corrected pgrba Files
- GEMPAK Metafiles For HPC Medium Range Desk Created in Production

## First Implementation of NAEFS – Summary

- Bias corrected members of joint MSC-NCEP ensemble
- Decaying accumulated bias (~past 50 days) for each var. for each grid point
- For selected 35 of 50 NAEFS variables
- 32(00Z), 15(06Z), 32(12Z) and 15(18Z) joint ensemble members
- Bias correction against each center’s own operational analysis
- Weights for each member for creating joint ensemble (equal weights now – unequal weights to be added later)
- Weights don’t depend on the variables
- Weights depend on geographical location (low precision packing)
- Weights depend on the lead time
- Climate anomaly percentiles for each member
- Based on NCEP/NCAR 40-year reanalysis
- Used first 4 Fourier modes for daily mean,
- Estimated climate pdf distribution (standard deviation) from daily mean
- For selected 19 of 50 NAEFS variables
- 32(00Z), 15(06Z), 32(12Z) and 15(18Z) joint ensemble members
- Adjustment made to account for difference between oper. & re-analysis
- Provides basis for downscaling if local climatology available
- Non-dimensional unit

## Bias Correction Method & Application    

-  decaying averaging mean error = (1-*w*) * prior t.m.e + *w* * (f – a)

-   Bias Correction: application to NCEP operational ensemble 15 members

- Bias Correction Method & Application**   **

- 6.6%

- 3.3%

- 1.6%

*For separated cycles**, **each lead time and individual grid point, t.m.e = time mean error*

-  Test different decaying weights.
-    0.25%, 0.5%, 1%, 2%, 5% and
-    10%, respectively
-   Decide to use 2% (~ 50 days)
-    decaying accumulation bias
-    estimation

*Toth, Z., and Y. Zhu, 2001*

## List of Variables for Bias Correction, Weightsand Forecast Anomalies for CMC & NCEP Ensemble

## NCO parallel for GEFS and statistic results

- Start from 01/15/2006 (testing)
- 14 perturbed runs and control for each cycle
- With new file structures
- Start from 02/01/2006 (real time parallel)
- Adding ET scheme (03/07/2006)
- Start from 04/10/2006 (NAEFS post products, partly)
- Use GEFS new system (May implementation)
- Create bias corrected forecast
- Forecast anomalies
- NCO real-time parallel verification statistics are posted at:
- _[http://](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[wwwt](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[.](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[emc](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[.](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[ncep](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[.](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[noaa](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[.](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[gov](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[/](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[gmb](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[/](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[yzhu](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[/html/](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[opr](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[/](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[prx](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)__[_daily.html](http://wwwt.emc.ncep.noaa.gov/gmb/yzhu/html/opr/prx_daily.html)_ (available now)
- Updated every morning
- Retrospective experimental verification statistics are posted at:
- _[http://www.emc.ncep.noaa.gov/gmb/yzhu/html/opr/et14m_daily.html](http://www.emc.ncep.noaa.gov/gmb/yzhu/html/opr/et14m_.html)_ (available now)
- Updated, finished
- Participated Evaluations

## Summary of Retrospective and Parallel Runs

- Period:
- 08/20/2005 – 09/30/2005 (retrospective runs)
- 03/01/2006 – 04/26/2006 (NCO real time parallel)
- Statistics for
- Hurricane track errors (retrospective runs only)
- Atlantic-, East Pacific-, West Pacific- basins, total basins
- RMS errors and AC scores for ensemble mean
- NH and SH ex-tropic
- Probabilistic verifications (ROC)
- NH and SH ex-tropic
- Conclusions
- Tropical  –  mean of track error (slightly improved)
- Improved (48-, 72-, 96-hours over all)
- Northern hemisphere
- Mean -improved from retrospective runs;-similar from NCO real time
- Probabilistic (improved)
- Southern hemisphere
- Mean –similar from retrospective runs; -improved from NCO real time
- Probabilistic (improved)

## Hurricane Track Errors (Period: 08/20-09/30/2005)

**Cases      174      157      141      128      101       75         48**

**Hours**

- _ENSs-operational ensemble_
- _ENSx-retrospective runs_
- _GFSs-operational GFS_

_**Atlantic Basin**_

**Cases      181      165     149      135      109       85         69**

**Cases      177      161      145      129      101       66         44**

**Cases      532      483      435      392      311       226      161**

_**All Basins**_

_**West Pacific Basin**_

_**East Pacific Basin**_

## Ensemble mean evaluations

- _Improving skills from/after day 3_

- _Much better than GFS after 72 hours_

- _65% AC scores – useful skill_
- _Ens. extended add. 20 hours_

- _ENS_s – opr. ensemble_
- _ENS_x – retrospective runs_

- _Improved_

- _RMSs are very similar_

- _RMSs are very similar_

- _ENS_s – opr. ensemble_
- _ENS_x – real time parallel_

- Northern Hemisphere

- Southern Hemisphere

- Right – Real Time Parallel

- Left – Retrospective Runs

## Probabilistic Evaluation (ROC)

- _ENS_s – operational ensemble_
- _ENS_x – retrospective runs_

- _ENS_s – operational ensemble_
- _ENS_x – real time parallel runs_

- _Improved_

- _Improved_

- _Improved_

- _Improved_

- Northern Hemisphere

- Right: Real Time Parallel

- Left: Retrospective Runs

- Southern Hemisphere

## Summary of NAEFS First Implementation

- Period:
- 04/10/2006 – Current (NCO real time parallel)
- Maps comparison for bias (before and after)
- 500hPa height, 2m temperature
- Statistics for
- Bias reduction in percentage
- Height, temperature, winds
- RMS errors
- Probabilistic verifications (ROC)
- NH, SH and tropic
- Conclusions
- Bias reduced (approximately 50% at early lead time)
- RMS errors improved by 9% for d0-d3
- Probabilistic forecast
- Improved for all area, all lead time
- Typically for NH, 20-24 hours improvement from d7

## 500hPa height: 120 hours forecast (ini: 2006043000)

- Shaded:      left – raw bias                          right – bias after correction

## 2 meter temperature: 120 hours forecast (ini: 2006043000)

- Shaded:     left – raw bias                          right – bias after correction

## Bias Improvement (absolute value) after Bias correction

- Overall bias reduction:
- (globally)
- D0-3: 50%
- D3-8: 40%
- D8-15: 30%

- _500hPa height_

- _850hPa temperature_

- _Sea level pressure_

- _2m Temperature_

- There is daily variation after bias correction, more bias reduced for valid 12Z cycle

## Bias Improvement (absolute value) after Bias correction

- Overall bias reduction:
- (Tropic)
- D0-3: 50%
- D3-8: 45%
- D8-15: 40%

- _10m V-component_

- _2m temperature_

- _Sea level pressure_

- _10m U-component_

## Evaluation after bias correction (16 cases)

- _Northern Hemisphere_

- _Tropics_

- _Southern Hemisphere_

**Black-operational ensemble (10m)**
**Red-real time parallel ensemble (14m)**
**Green-real time parallel ensemble after**
**bias correction (14m)**

- _RMS errors for ensemble mean reduced for 48-h forecast (~9%)_

- _Probabilistic skill_
- _Extended 20-h for d-7_

## NAEFS Performance Review

**KEY PERFORMANCE MEASURES**

** **
_** **_

## NAEFS Configuration Review (NCEP)

**MINIMAL (PREFERRED) CONFIGURATION FOR THE GLOBAL ENSEMBLE FORECAST SYSTEMS OPERATIONAL AT CMC AND NCEP**
** **

## Summary of Subjective Analysis

- HPC
- Noted Two Month Evaluation Window Was Helpful and Better Evaluation Data Availability to HPC Compared to Previous Evaluations
- Noted Parallel Mean Seemed to Hold Onto a Stronger MSLP Solution Well in the Medium Range Than Operational Mean
- Individual Members Tended to Cluster on a Solution More Than Operational Version
- Improved International Desk Confidence in Parallel Forecast of Certain Patterns in Southern Hemisphere
- CPC
- Based on Limited Set of Parallel Runs for D+8 and D+11 Forecasts, Anomaly Correlation Skill Scores Between Parallel and Operational System Were Similar
- Slight Improvement in Score for the Southern Hemisphere Extratropics
- OPC
- Proposed Changes are Roughly Equal to the Current System for the Northern Hemisphere
- On the Day to Day Noticed a Tendency for the Parallel Mean SLP Field to Better Cluster with Low Positions.  Therefore in the Mean Ocean Cyclones Tended to Have Lower Pressure than the Operational Mean and More Resemble a Single Model Solution
- Basically, the Mean Fields Appear More Realistic and Resemble Synoptic Scale weather Systems
- SPC
- Evaluated Primarily as Guidance for the Experimental Day 4 to 8 Severe Weather Outlook
- Probabilistic Forecasts of Combined Severe Weather Parameters were Generally Equal or Better than Operational
- 6-Hourly Output Very Helpful
- Minimal (subjective) Differences Were Noted in Ensemble Mean Products

## Summary of Subjective Analysis

- TPC
- The reruns from last year's hurricane season showed no significant degradations, and some slight improvements, to tropical cyclone track forecasts in the Atlantic basin in retrospective reruns.
- Although there was some degradation of track forecasts for the eastern North Pacific, the sample for that basin was quite small.
- The results from a global verification of track forecasts from that same period showed no significant degradations.
- NWS KCTP
- Detailed 11-12 May 2006 Case Study
- Additional Members Were Great but the Anomaly Files Hold Incredible Promise.  Huge Potential in These Data Just to Show the Probability of an Anomalous Forecast.  **This Relates Directly to the Emergent Focus in the NWS to Identify the “Significant” Events.**
- See a Bright Future in More Probabilistic “Alarm Bell” Maps Warning Forecasts of the Potential for Anomalies Associated with Big Precipitation Events, Snow Storms, etc.
- General Improvement with Additional Members
- New File Naming Conventions With P1-14 Versus P1-5 and N1-5 Made Data Easier to Work With
- Air Force Weather Agency
- Began Using Parallel Data Feed at End of April So Subjective Experience with Data Limited
- Pleased with Switch from Breeding Modes to Ensemble Transform Initial Conditions
- Acknowledgement of Data Arriving Incrementally as Forecast Runs Versus Distribution of Products at End of Forecast
- To Effectively Support Joint Global Ensemble (JGE) Products and Running of Limited Area Ensembles AFWA and JEFS Still Need
- More Data in PBL and Stratosphere
- Horizontal Resolution needed to Increase to 0.5 Deg or Better

## NAEFS Product Distribution Plans

- Basic products: Bias corrected members, weights, climate percentiles
- NCEP Service Centers – NAWIPS files
- External users - ftp sites – GRIB2 distribution planned?
- End Products
- NCEP Service Centers: Interactive access via NAWIPS ensemble functionalities
- NAWIPS ensemble demonstration planned after new NAWIPS release (22 May)
- Scheduled product generation based on compiled request list, start ~ Dec 2006
- NCEP Service Centers – NAWIPS graphics
- Web distribution: Gif images onto NCEP operational web site
- NDGD Plans
- Distribute bias corrected ensemble information on 1x1 lat/lon grid
- 10, 50, 90 percentile values in NAEFFS forecast distribution (3 fields)
- Associated climate percentiles (3 additional fields for downscaling applications)
- Downscale bias corrected NAEFS members using RTMA analysis
- Determine systematic difference (downscaling vector) between GFS and RTMA analyses
- Distribute downscaling vector on NDGD grid

## Background !!!!!

## RAW & BASIC PRODUCT AVAILABILITY

*2005, **2006**, **2007**, **2008*

## END PRODUCTS

- End product generation
- Can be center specific
- Need to conform with procedures/requirements established at different centers
- End products generated at NCEP
- Based on prioritized list of requests from NCEP Service Centers
- Graphical products (including Caribbean, South American, and AMMA areas)
- NCEP official web site (gif – NA, Caribbean, SA, AMMA)
- NCEP Service Centers (NAWIPS metafile)
- Gridded products
- NAWIPS grids
- NCEP Service Centers (list of 661 products)
- GRIB2 format
- Products of general interest (Possible ftp distribution, no decision yet on products)
- NDGD (10-50-90 percentile forecast value + associated climate percentile)
- End products generated at MSC
- TBD
- End products generated jointly
- Experimental probabilistic Week-2 forecast
- Fully automated, based on basic products: bias corrected, weighted climate anomalies
- Can become official product once performance reaches current operational level

## ENSEMBLE FUNCTIONALITIES

|  | FUNCTIONALITY | CENTRALLY GENERATED | LOCALLY GENERATED | INTERACTIVE
ACCESS |
| --- | --- | --- | --- | --- |
| 1 | Mean of selected members  Done |  |  |  |
| 2 | Spread of selected members Done  |  |  |  |
| 3 | Median of selected values Done Sept. 2005 |  |  |  |
| 4 | Lowest value in selected members Done Sept. 2005 |  |  |  |
| 5 | Highest value in selected members Done Sept. 2005 |  |  |  |
| 6 | Range between lowest and highest values Done Sept. 2005 |  |  |  |
| 7 | Univariate exceedance probabilities for a selectable threshold value Done, Dec 05 |  |  |  |
| 8 | Multivariate (up to 5) exceedance probabilities for a selectable threshold value Done, Dec 05 |  |  |  |
| 9 | Forecast value associated with selected univariate percentile value Done Sept. 2005 |  |  |  |
| 10 | Tracking center of maxima or minima in a gridded field (eg – low pressure centers) Done Sept. 2005 |  |  |  |
| 11 | Objective grouping of members Planning starts FY06, Deliver FY07-08 |  |  |  |
| 12 | Plot Frequency / Fitted probability density function at selected location/time (lower priority) Detailed Planning FY06, Deliver FY07 |  |  |  |
| 13 | Plot Frequency / Fitted probability density as a function of forecast lead time, at selected location (lower priority) Detailed Planning FY06, Deliver FY07 |  |  |  |
| 14 | Spaghetti (ability to interactively change contour/domain etc)  Basic function done; Interactive version to be scheduled (TBS) |  |  |  |

- List of centrally/locally/interactively generated products required by NCEP Service Centers for each functionality are provided in attached tables (eg., *MSLP, Z,T,U,V,RH, etc, at 925,850,700,500, 400, 300, 250, 100, etc hPa)*

***Potentially useful functionalities that need further development:***
- - Mean/Spread/Median/Ranges for amplitude of specific features *(TBS)*- Mean/Spread/Median/Ranges for phase of specific features *(TBS)*

***Additional basic GUI functionalities:**** *- Ability to manually select/identify members *(TBS)*
- - Ability to weight selected members *Done, Sept. 05*

## ENSEMBLE PRODUCT REQUEST LIST

- NCEP SERVICE CENTERS, OTHER PROJECTS

## NDGD FORECAST UNCERTAINTY - DOWNSCALING

- Ensemble uncertainty information
- Sent on NDGD grid for convenience (if no big overhead)
- Valid on model grids (32km for regional, 110 km for global ensemble)
- How to bridge gap between model and NDGD grids?
- Anomaly uncertainty information – proposed methodology
- Establish reanalysis climatology
- In progress for global (NAEFS), methods can be transferred to regional reanalysis
- Bias correct ensemble forecasts (wrt operational analysis)
- Take 10-50-90 percentile values from bias corrected ensemble
- (For establishing anomaly forecasts, adjust 10-50-90 percentile values to look like re-analysis)
- Check climatological percentile corresponding to 10-50-90 forecast percentiles
- Provide climatological percentiles corresponding to 10-50-90 percentile forecast values as second set of guidance products

## Early studies for ET

- Northern Hemisphere

- Southern Hemisphere

- Tropical

_**Early studies for ET**_
- Winter of 2002-2003
- ROC scores for 32 cases
- ENS-o  control runs
- ENS-s  ET-20 members
- ENS-x  ET-10 members

## Perturbation versus Error Correlation Analysis (PECA)

- Retrospective runs

- NCO parallel runs

## Tropical scores for

- _There is no big effect in Tropical by apply ET in generally_

- _Probabilistic skills_

_**Tropical scores for**_
_**NCO real time parallel**_

_**Ensemble mean**_

_**Ensemble probabilistic**_

## Track errors and spreads2004 Atlantic Basin (8/23-10/1)

- From Timothy Marchok (GFDL)

- Reduced mean track errors and spreads

## Hurricane track errors2 basins (Atlantic and e-Pacific)

- Track errors (miles)

- Percentage improvement to operational ensemble

- Period: 20040824-20040930 (53-103 cases)

## Bias Improvement (absolute value) after Bias correction

- Overall bias reduction for Northern Hemisphere

- _Bias – before/after bias correction_

## RMS errors before/after bias correction

- _Opr. ROC scores for past 5 years_

## RPSS  before/after bias correction

- _RPSS performance for past 5 years_

## Update of NAEFS status from MSC

- Data exchange
- NCEP Grib1 data are received correctly at our end. We still are missing the NCEP analyses files pushed towards us. The reanalysis files are also missing, but it isn't as critical. At the moment, we're not able to produce debiased NCEP EPS fields.
- We should plan to exchange the debiased fields on regular basis. This will ensure that we have correctly implemented the scripts on both sides.
- A point was mentioned about the necessity to archive EPS files. We currently are archiving EPS MSC files and will archive the debiased field as well. We are planning to archive the NCEP EPS, but it will be for our own needs.
- The direct link between NCEP and MSC should be reexamined.
- Fields/variables
- We should aim in having TIGGE and NAEFS variables as close as possible.
- Plans
- Plans for MSC EPS for 2007:
- 20 members
- 1 X 1 resolution
- Only GEM model will be used
- CMC analysis:
- low resolution mean Kalman analysis
- Others
- Redundancy is necessary for contingency.
- In order to plan for weighting algorithm, we should develop a testbed for weighting methodology.