# Presentation: 114212

## Case Study and Model Performance Part II: General GFS/GENS model performance

- William R. Bua, UCAR/COMET
- Session 6, 8:00 a.m. Friday 12 October 2007

## Outline

**Objectives: **
- GFS (from _[http://](http://meted.ucar.edu/)__[meted.ucar.edu](http://meted.ucar.edu/)_) model architecture
- Sources of error for GFS
- The NAEFS
- CMC GEM model architecture
- GEFS configuration
- NAEFS configuration
- Verification scores
- General discussion of limitations of ensemble prediction systems and predictability

## Operational and Ensemble GFS configuration

- Dynamics
- Physics
- Precipitation and clouds
- Land surface physics
- Vegetation
- Soil moisture
- Planetary boundary layer
- Radiation (and relationship with precipitation and clouds)
- Free atmosphere turbulence

## GFS Dynamics

- Equations of motion (advection, continuity) calculated in spectral space (sines and cosines)
- Exact derivatives for all wave numbers (1/wavelength) calculated
- Truncation error results from limiting the maximum wave number (minimum wavelength) for calculations
- Operational T382 (~50 km) through 180 hours, T190 (~100 km) from 180-384 hours
- Number of waves for ensemble T126 through 384 hours (~ 150 km)

## GFS Dynamics

- Vertical coordinate
- Sigma-pressure hybrid
- Levels placed as at right
- Advantage of hybrid sigma-pressure:
- Sigma-pressure reduces pressure gradient calculation errors resulting from tilted sigma surfaces above about 500-hPa considerably
- Has reduced wind errors downwind of Himalayan massif by about 5-10%

## GFS Physics Grid

- The physics grid
- Sub-grid scale physical process calculations done at grid points and transformed into “spectral space”
- Grid is 0.31°-0.38° resolution over southern Africa domain for operational GFS, about 0.9°-1.1° resolution for ensemble GFS
- Topography
- T382 topo at right
- Highest point resolved is in Lesotho (2725-m)
- T126 topo next
- Highest points in Kenya and Lesotho (2096-m)
- Land-sea mask
- T126 loses islands, some lakes, shoreline resolution

- T382

## GFS Dynamics

- Precipitation and clouds
- “Grid-scale precipitation”
- Simple microphysical processes are modeled (“simple cloud”)
- Precipitation hydrometeors NOT tracked; fall out instantaneously
- Cloud water (in both liquid and solid phases) is tracked and used to determine radiative qualities of cloud
- Convective scheme
- Simplified Arakawa-Schubert (SAS)
- Physically realistic, includes observed convective processes
- Cloud fraction determined by precipitation rate, cloud depth by lifted condensation level (bottom) and level of neutral buoyancy (top)

- T382

## GFS Vegetation Type and Fraction

- Vegetation type and greenness fraction
- Taps into sub-surface soil moisture
- 13 vegetation types
- Climatological seasonal cycle for green vegetation fraction
- Vegetation canopy can retain up to 2-mm of water and drip-through is modeled
- Greenness fraction taken from climatology (NDVI)
- If excessive drought or wetness, incorrect active/green vegetation may result in surface energy balance problems

## Veg fraction Jan-Apr

## Veg fraction, May-Aug.

## Veg fraction, Sep.-Dec.

## GFS Soil Model

- Soil moisture model
- Surface layer (0-10cm)
- Root zone layer 1 (10-40cm)
- Root zone layer 2 (40-100 cm)
- Deep soil layer (100-200 cm)
- For moisture, diffusion and gravitational force act on sub-surface, depends on soil properties
- Soil thermal model
- Additional layer (200-800 cm) with deep soil temp (~avg annual temperature) constant (bottom boundary condition)
- Diffusion of heat through layers with top boundary condition provided by surface temperature

## Soil Types

## GFS Radiation

- Short wave (Chou, 1990, 1992)
- Predicted ozone (O3) and water vapor (H2O)
- Prescribed CO2
- Prescribed O2
- Aerosols
- RRTM long wave
- CO2, H2O, O3, CH4, N2O, CCl4, chloro-fluorocarbons

## GFS Radiation and Clouds

- Cloud radiative properties depend on water phase (liquid or solid), cloud water mixing ratio
- Cloud fraction depends
- For grid-scale clouds, on cloud water mixing ratio and RH
- For convective cloud, on convective precipitation amount
- Clouds are overlapped randomly

## GFS surface layer

- Transport of heat and moisture in surface layer (treated as 1st model layer) depends on vertical gradients and winds
- Surface roughness affects the wind speed and is set based on vegetation type
- Gradient of pot temp *θ*, specific humidity *q*, *u* and *v* wind velocity determines sensible, latent heat, and momentum fluxes

## GFS Planetary Boundary Layer and Free Atmosphere Turbulence

- A “non-local scheme”
- PBL top set to where Bulk Richardson number Ri is first > 0.5
- Vertical diffusion coefficient is fit to flux at PBL top and surface, then formula used to determine diffusion rate through the PBL
- In free atmosphere, local wind shear and stability determine turbulent vertical transports

- Ri >0.5

- Ri ≤0.5

## Outline

**Objectives: **
- GFS (from _[http://](http://meted.ucar.edu/)__[meted.ucar.edu](http://meted.ucar.edu/)_) model architecture
- Sources of error for GFS
- The NAEFS
- CMC GEM model architecture
- GEFS configuration
- NAEFS configuration
- Verification scores
- General discussion of limitations of ensemble prediction systems and predictability

**Questions ?**

## GFS-Specific Sources of error

- Data assimilation/Grid point statistical interpolation (GSI)
- Bad model 1st guess (the 6-hour forecast)
- Extreme excursions from balance constraint used in the GSI (data might be right, but will be rejected because the 1st guess is always assumed to be good)
- Lack of time interpolation of data to analysis time
- Coarseness of some data (e.g. satellite radiances in the vertical)

## GFS-Specific Sources of error

- Data assimilation system
- Analysis system assumptions
- The background field/first guess (6-hour GFS forecast) is of good quality
- Innovations/corrections to the background are spread symmetrically around the innovation
- Not true in areas with vertical or horizontal gradients
- Satellite radiances (important over oceans) are coarse in vertical
- Depends on background field/first guess for placement of innovations
- Dynamics
- Some flows cannot be forecast well by the GFS
- Strong height gradients
- Sharply curved flow
- Blocked flow (especially onset and decay of blocking)
- Cutoff lows

## GFS-Specific Sources of Error

- Physics
- Convective parameterization
- Convective parameterization removes instability at sub-grid scales so that the grid-scale scheme does not attempt to remove it
- Convective parameterization does not always remove enough, especially for elevated convective instability
- When the grid-scale precipitation scheme tries to remove instability, result is precipitation “bulls-eyes” that look a little bit like mesoscale convective systems (MCSs)
- Often there are MCSs in the vicinity....but not where they are placed by GFS

## GFS-Specific Sources of Error

- Topography (Orographic precipitation? Errors of representativeness for locales in areas of rough terrain?)
- Surface energy balance considerations
- Soil moisture somewhat constrained (relaxes toward long-term NCEP climatology)
- Climatological vegetation fraction (will not accurately reflect anomalously low or high greenness resulting from drought or moist periods, respectively)

## Outline

**Objectives: **
- GFS (from _[http://](http://meted.ucar.edu/)__[meted.ucar.edu](http://meted.ucar.edu/)_) model architecture
- Sources of error for GFS
- The NAEFS
- CMC GEM model architecture
- GEFS configuration
- NAEFS configuration
- Verification scores
- General discussion of limitations of ensemble prediction systems and predictability

**Questions ?**

## The Canadian Meteorological Center Global Environmental Model (GEM)

- Dynamics
- Grid point 0.9° L28 (~100 km resolution)
- Sigma type terrain following coordinate.
- Solar and Infrared Radiation:
- Interact with water vapour, ozone, carbon dioxide and clouds.
- Land Surface Scheme:
- Known as “ISBA” (Interactions *between the Soil, Biosphere and Atmosphere)*. Includes prognostic variables for surface and soil temperatures, snow depth, and soil volumetric water contents.
- PBL Turbulence:
- Turbulence in the planetary boundary layer is achieved through vertical diffusion, with the diffusion coefficients based on stability and kinetic energy.

## The Canadian Meteorological Center Global Environmental Model (GEM)

- Surface Layer:
- Based on Monin-Obukhov similarity theory
- Shallow Convection Scheme:
*Non-precipitating *, based on large scale convergence of moisture.
- Deep Convection Scheme:
- Kain-Fritsch
- Stratiform Precipitation:
- Sundqvist condensation scheme (predicted cloud water)
- Sub-Grid Scale Orography:
- Gravity Wave Drag
- Emulates the breaking of mountain waves in the stratosphere
- Affects the winds at relatively high levels (~100 hPa)
- Not a large factor in short-term integrations, but can be important in the medium term.
- Blocking Term:
- Reduces low-level winds in regions of rough terrain.
- Result in a displacement of precipitation patterns upstream of the mountain ranges

## GFS Ensemble Forecast System (GEFS)

- Initial condition perturbations produced by Ensemble Transform Bred Vectors (ETBV)
- Flow-dependent initial perturbations determined through statistical methods
- 4 cycles per day (00, 06, 12, 18 UTC)
- 20 ensemble perturbations and one control (21 members in all)
- NO MODEL PERTURBATIONS (all ensemble members run from most recent version of GFS, but at coarser resolution)

## Canadian EPS

- Members:
- 16+1 members:
- GEM 1.2° L28 + SEF T149 L28 (~150 km resolution).
- 16-day integration.
- Twice a day (00 and 12 UTC).
- Simulation of initial condition uncertainties:
- Perturbed ensemble Kalman filter data assimilation.
- Simulation of model uncertainties:
- A multi-model approach, each member having its own physics parameterization.

- Old – As of January 2007

**Notes:**
Here is a summary of the main characteristics of the Canadian Ensemble Prediction System (old version) as of January 2007. 

## Canadian EPS

- Members:
- 20+1 members:
- GEM 0.9° L28 (~100 km resolution).
- 16-day integration.
- Twice a day (00 and 12 UTC).
- Simulation of initial condition uncertainties:
- Perturbed ensemble Kalman filter data assimilation.
- Simulation of model uncertainties:
- A multi-model approach, each member having its own physics parameterization.
- Stochastic perturbations added to tendencies in the parameterized physical processes.

- New – To come

**Notes:**
Here is a summary of the main characteristics of the Canadian Ensemble Prediction System - new version planned to become operation by in the third quarter of 2007. The Canadian EPS includes perturbation of observations and data assimilation cycles to simulate uncertainties related to initial conditions. The Canadian EPS is also a multi-model ensemble with stochastic Physics to simulate uncertainties related to numerical modeling of the atmosphere.

## Canadian EPS

- A. Erfani, CMC

**Notes:**
This graph represents the Ensemble Prediction System (new version) in terms of resolved scales by the models. Ideally, the members of an Ensemble Prediction System should have a resolution comparable to that of the deterministic model, in order to be in a better position to have a sound judgment on the reliability of the deterministic model in the problem at hand.

## North American Ensemble Forecast System (NAEFS)

- Combines GEFS and CEFS for statistical analyses
- Combination will be multi-model with flow dependent perturbations (similar methods in both models)
- Bias correction of each model, done separately against its own analyses
- 40 members and 2 controls at 00 and 12 UTC (CEFS will go to 4 times per day soon)

## Impact of Merging CEFS and GEFS

- 5-day forecast of climate anomalies from 00 UTC 09 September 2007:  GEFS

## Impact of Merging CEFS and GEFS

- 5-day forecast of climate anomalies from 00 UTC 09 September 2007:  NAEFS

## Outline

**Objectives: **
- GFS (from _[http://](http://meted.ucar.edu/)__[meted.ucar.edu](http://meted.ucar.edu/)_) model architecture
- Sources of error for GFS
- The NAEFS
- CMC GEM model architecture
- GEFS configuration
- NAEFS configuration
- Verification scores
- General discussion of limitations of ensemble prediction systems and predictability

**Questions ?**

## Large-Scale scores used

- Anomaly Correlation (AC)
- Spatial correlation of forecast and observation departures from climatology
- Bias
- Difference in means for forecasts versus observations
- Root-mean-square error (RMSE)
- where *f* and *o* are the forecast value and observation value, respectively over a domain of N points

## Long-term Performance at Hemispheric Scales:  GFS

- 5-day anomaly correlation scores for 500-hPa heights
- Baseline is purple (CFS, frozen)
- Shows “predictability” of atmosphere
- Increase over purple line  impact of improvements
- GFS black, ECMWF blue
- Verification against respective analyses
- N Hemisphere (NH) and S Hemisphere (lower panel, red box) ECMWF scores approx. equal
- GFS lower skill in SH
- Strong improvement in SH 1999-2003 as satellite assimilation improved (methods, quantity/quality)

- through August 2007

## Unsmoothed AC over last 5 years: GFS

- Seasonality of predictability much more pronounced in NH than SH
- Skill generally better in NH than SH on average
- Note SH includes most of Antarctica

## Most recent period: GFS 

## Most recent period “dieoff curves”: GFS

- Planetary waves

- Long waves

- Short waves

- }

- }

- }

-   Dieoff curves show reduction in AC over time for waves
-   Graphic for Sept 2007 for Southern Hemisphere
-   Short waves lose skill   most quickly (day 3.5   or so for GFS, 4.5 for   ECMWF and UKMET)
-   Longer waves lose    skill at day 6 for GFS,   day 7 for ECMWF)
-   Planetary waves have   best skill (past day 9    for ECMWF, around day   8.75 for GFS)

## Bias scores: GFS for SH, winter ‘07

- Bias (mean forecast – mean verification)
- Gives sign of error (low or high)

- 700-hPa temperture

- 850-hPa temperture

- 1000-hPa temperture

## GENS performance in S Hemisphere

- Rank Histogram/ Talagrand diagram
- 30 mbrs from spring ‘07 (15 mbrs for GENS and 15 from CENS; 30n is raw, 30nb is bias-corrected)
- Optimal % is 1/30 or 3.33%
- Day 1:  very little consistent difference (underdispersive)
- Day 3:  improvement from bias-corr is consistent, throughday 15
- Underdispersive tendency throughout

- 1 day

- 3 day

- 5 day

- 8 day

- 12 day

- 15 day

- cold bias

## Brier Skill Score, Ens mean error, mean absolute error GEFS/CEFS/NAEFS SH fall ‘07

- 2-m temperature for NCEP/CMC separately, and then combined ensemble systems
- Significant improvements in scores from combining two EPS in ens mean absolute error, resolution

## Brier Skill Score (CEFS/GEFS/NAEFS)

- BSS		Higher better
- Resolution	Higher better
- Reliability		Lower better
- 2-m temperature with and without bias correction, fall ‘07

- CEFS

- GEFS

- NAEFS

## GFS performance in southern Africa

- Local verification depends on having data with which to verify!
- Africa desk will work with NCEP to start keeping verification statistics over southern Africa once there is such data
- Watch the Africa desk webpage for verification graphics on precipitation, heights, winds, temperatures once the data becomes available.

## Outline

**Objectives: **
- GFS (from _[http://](http://meted.ucar.edu/)__[meted.ucar.edu](http://meted.ucar.edu/)_) model architecture
- Sources of error for GFS
- The NAEFS
- CMC GEM model architecture
- GEFS configuration
- NAEFS configuration
- Verification scores
- General discussion of limitations of ensemble prediction systems and predictability

**Questions ?**

## Predictability and the GFS/GENS/NAEFS

- The atmosphere is chaotic and loses predictability in the current state of the art NWP models at an average of about 7 days
- Best practices based in sound science are to use probabilistic methods to develop a forecast and express the forecast to users probabilistically
- EPS (GENS and new NAEFS) are the best tools for doing probabilistic forecasting

## Predictability and the GFS/GEFS/NAEFS

- Predictability of the atmosphere is dependent on
- Flow regime
- Stable regimes more predictable, regime transitions are more troublesome
- Does the flow regime in your area look like what you’d expect, given the remote forcing present (El Nino, La Nina, and so on)?
- Season of year
- Warm season less predictable typically than cold season
- Convective type regimes (typical of the tropics, heavy reliance on parameterization) less predictable than strongly forced (e.g. extratropical baroclinic wave) regimes
- Scale of features
- Large-scale features generally more predictable than small-scale features

## Deterministic forecast has less skill in the long run than either the individual or combined ensemble means, especially after day 3
Gains one day at 0.6 skill threshold

- Gains one day at 0.6 skill threshold

- Predictability and the GFS/GEFS/NAEFS

- Ens mean beats T382 GFS past day 2

- traditional skill/no skill AC threshold
- AC = 0.60
- Gain about 1 day of skill

## Predictability and the GFS/GEFS/NAEFS

- Ensemble forecasts have systematic biases because the models on which they are based *also* have systematic biases
- Verification procedures can reveal these biases so that correction can be applied
- Bias correction can help with this somewhat ... but will not remove all bias
- Correction based on recent past performance
- Current situation will almost always be different to some degree

## Follow-Up on the Training

- The Africa Desk and COMET are available to help with training on NWP and EPS
- Make use of existing training materials
- Problem with relevance to local regions
- There **may** be additional work done of relevance to your countries of origin ... but at MINIMUM ....
- We will be packaging the materials from this workshop for you to use for your forecasters, probably during summer 07-08 ... stay tuned for more details

## So Long ... For Now

- I feel privileged to have had the opportunity to work with you!
- Feel free to contact me with any questions you may have on any of the products or systems that were brought up in this workshop
- Bill.Bua@noaa.gov