# Presentation: 178678

## ENSEMBLE FORECASTING AT NCEP

***Zoltan Toth******(3)******,**** *
*Yuejian Zhu**(4)**, Jun Du **(4)**, Richard Wobus **(4)**, Tim Marchok, Mozheng Wei**(5)**, *
*Ackn.: S. Lord **(3)**, H.-L. Pan **(3)**, R. Buizza**(1)**, P. Houtekamer**(2)**, S. Tracton **(6)*
*(1)** : European Centre for Medium-Range Weather Forecasts, Reading UK (www.ecmwf.int)*
*(2)** : Meteorological Service of Canada, Dorval, Quebec, Canada  (www.msc-smc.ec.gc.ca)*
*(3)** : NCEP/EMC, Washington, US (*_[www.emc.ncep.noaa.gov](http://www.emc.ncep.noaa.gov/)_*)*
*(4)** : SAIC at NCEP/EMC, Washington, US (www.emc.ncep.noaa.gov)*
*(5)** : UCAR Visiting Scientist, NCEP/EMC, Washington, US*
*(6)** : ONR*

## OUTLINE

- MOTIVATION FOR ENSEMBLE/PROBABILISTIC FORECASTING
- User Needs
- Scientific needs
- SOURCES OF FORECAST ERRORS
- Initial value
- Model formulation
- ESTIMATING & SAMPLING FORECAST UNCERTAINTY
- DESCRIPTION OF NCEP ENSEMBLE FORECAST SYSTEMS
- Global
- Regional
- Coupled ocean-atmosphere
- FORECAST EXAMPLES
- VERIFICATION
- ONGOING RESEARCH / OPEN QUESTIONS

## MOTIVATION FOR ENSEMBLE FORECASTING

***FORECASTS ARE NOT PERFECT - *****IMPLICATIONS FOR:**
**USERS:**
- Need to know how often / by how much forecasts fail
- Economically optimal behavior depends on
- Forecast error characteristics
- User specific application
- Cost of weather related adaptive action
- Expected loss if no action taken
- EXAMPLE: Protect or not your crop against possible frost
- Cost = 10k, Potential Loss = 100k => Will protect if P(frost) > Cost/Loss=0.1
***NEED FOR PROBABILISTIC FORECAST INFORMATION***
**DEVELOPERS:**
- Need to improve performance   - 	*Reduce error in estimate of first moment*
- Traditional NWP activities (I.e., model, data assimilation development)
- Need to account for uncertainty -	*Estimate higher moments*
- New aspect – How to do this?
-  Forecast is incomplete without information on forecast uncertainty
***NEED TO USE PROBABILISTIC FORECAST FORMAT***

**USER NEEDS – PROBABILISTIC FORECAST INFORMATION FOR MAXIMUM ECONOMIC BENEFIT**

## SCIENTIFIC NEEDS - DESCRIBE FORECAST UNCERTAINTY ARISING DUE TO CHAOS

*Buizza 2002*

## FORECASTING IN A CHAOTIC ENVIRONMENT

***DETERMINISTIC APPROACH - PROBABILISTIC FORMAT***
*** ***
**SINGLE FORECAST** -	* ****One integration with an NWP model***
-  Is not best estimate for future evolution of system
- Does not contain all attainable forecast information
-  Can be combined with past verification statistics to form probabilistic forecast
-  Gives **no estimate of flow dependent variations in forecast uncertainty**
**PROBABILISTIC FORECASTING** -	 ***Based on Liuville Equations***
-  Initialize with probability distribution function (pdf) at analysis time
-  Dynamical forecast of pdf based on conservation of probability values
-  **Prohibitively expensive** -
-  Very high dimensional problem (state space x probability space)
-  Separate integration for each lead time
-  Closure problems when simplified solution sought

## FORECASTING IN A CHAOTIC ENVIRONMENT - 2DETERMINISTIC APPROACH - PROBABILISTIC FORMAT

**MONTE CARLO APPROACH** –* ****ENSEMBLE FORECASTING***
* ****IDEA*****:****	****	****Sample sources of forecast error**
- Generate initial ensemble perturbations
- Represent model related uncertainty
* ****PRACTICE***:	**Run multiple NWP model integrations**
- Advantage of perfect parallelization
- Use lower spatial resolution if short on resources
-  ***USAGE*****:**		**Construct forecast pdf based on finite sample**
- Ready to be used in real world applications
- Verification of forecasts
- Statistical post-processing (remove bias in 1st, 2nd, higher moments)
***CAPTURES FLOW DEPENDENT VARIATIONS***
***	******	******	******	******	******	****** ******IN FORECAST UNCERTAINTY***

## SOURCES OF FORECAST ERRORS

***IMPERFECT KNOWLEDGE OF***
**INITIAL CONDITIONS**
-  Incomplete observing system (not all variables observed)
-  Inaccurate observations (instrument/representativeness error)
-  Imperfect data assimilation methods
-  Statistical approximations (eg, inaccurate error covariance information)
-  Use of imperfect NWP forecasts (due to initial and model errors) –
-  Effect of cycling (forecast errors “inherited” by analysis – use breeding)
**GOVERNING EQUATIONS**:
-  Imperfect model
-  Structural uncertainty (eg, choice of structure of convective scheme)
-  Parametric uncertainty (eg, critical values in parameterization schemes)
-  Closure/truncation errors (temporal/spatial resolution; spatial coverage, etc)
***NOTES****:*
-  Two main sources of forecast errors hard to separate =>
-  Very little information is available on model related errors
-  Tendency to attribute all forecast errors to model problems

## SAMPLING FORECAST ERRORS =

***REPRESENTING ERRORS ORIGINATING FROM TWO MAIN SOURCES***
**INITIAL CONDITION RELATED ERRORS**** – *****“Easy”***
** **Sample initial errors
-  Run ensemble of forecasts
-  It works
-  Flow dependent variations in forecast uncertainty captured (show later)
-  Difficult or impossible to reproduce with statistical methods
**MODEL RELATED ERRORS**** – *****No theoretically satisfying approach***
** **Change structure of model (eg, use different convective schemes, etc, MSC)
-  Add stochastic noise (eg, perturb diabatic forcing, ECMWF)
-  Works? Advantages of various approaches need to be carefully assessed
-  Are flow dependent variations in uncertainty captured?
-  Can statistical post-processing replicate use of various methods?
-  Need for a
-  more comprehensive and
-  theoretically appealing approach

## SAMPLING INITIAL CONDITION ERRORS

***CAN SAMPLE ONLY WHAT’S KNOWN – FIRST NEED TO***
***ESTIMATE INITIAL ERROR DISTRIBUTION***
***THEORETICAL UNDERSTANDING*** – THE MORE ADVANCED A SCHEME IS
- 					(e. g., 4DVAR, Ensemble Kalman Filter)
-  The lower the overall error level is
-  The more the error is concentrated in subspace of Lyapunov/Bred vectors
***PRACTICAL APPROACHES***** – **
**ONLY SOLUTION IS MONTE CARLO (ENSEMBLE) SIMULATION**
-  ***Statistical approach*** (dynamically growing errors neglected)
-  Selected estimated statistical properties of analysis error reproduced
-  Baumhefner et al – Spatial distribution; wavenumber spectra
-  ECMWF – Implicite constraint with use of Total Energy norm
*** ******Dynamical approach*** – Breeding cycle (NCEP)
-  Cycling of errors captured
-  Estimates subspace of dynamically fastest growing errors in analysis
-  ***Stochastic-dynamic approach*** – Perturbed Observations method (MSC)
-  Perturb all observations (given their uncertainty)
-  Run multiple analysis cycles
-  Captures full space (growing + non-growing) of analysis errors

## SAMPLING INITIAL CONDITION ERRORS

***THREE APPROACHES – SEVERAL OPEN QUESTIONS***
**RANDOM SAMPLING** – **Perturbed observations method** (MSC)
- Represents all potential error patterns with realistic amplitude
- Small subspace of growing errors is well represented
- Potential problems:
-  Much larger subspace of non-growing errors poorly sampled,
-  Yet represented with realistic amplitudes
**SAMPLE GROWING *****ANALYSIS***** ERRORS** – **Breeding** (NCEP)
- Represents dynamically growing analysis errors
- Ignores non-growing component of error
- Potential problems:
- May not provide “wide enough” sample of growing perturbations
- Statistical consistency violated due to directed sampling? Forecast consequences?
**SAMPLE FASTEST GROWING *****FORECAST***** ERRORS** – **SVs** (ECMWF)
- Represents forecast errors that would grow fastest in linear sense
- Perturbations are optimized for maximum forecast error growth
- Potential problems:
- Need to optimize for each forecast application (or for none)?
- Linear approximation used
- Very expensive

## ESTIMATING AND SAMPLING INITIAL ERRORS:THE BREEDING METHOD

**DATA ASSIM:** Growing errors due to cycling through NWP forecasts
**BREEDING:** - Simulate effect of obs by rescaling nonlinear perturbations
- Sample subspace of most rapidly growing analysis errors
- Extension of linear concept of Lyapunov Vectors into nonlinear environment
- Fastest growing nonlinear perturbations
- Not optimized for future growth –
- Norm independent
- Is non-modal behavior important?

## LYAPUNOV, SINGULAR, AND BRED VECTORS

**LYAPUNOV VECTORS (LLV):**
- Linear perturbation evolution
- Fast growth
- Sustainable
- Norm independent
- Spectrum of LLVs
**SINGULAR VECTORS (SV):**
- Linear perturbation evolution
- Fastest growth
- Transitional (optimized)
- Norm dependent
- Spectrum of SVs
**BRED VECTORS (BV):**
- Nonlinear perturbation evolution
- Fast growth
- Sustainable
- Norm independent
- Can orthogonalize (Boffeta et al)

## PERTURBATION EVOLUTION

**PERTURBATION GROWTH**
- Due to effect of instabilities
- Linked with atmospheric phenomena (e.g, frontal system)
**LIFE CYCLE OF PERTURBATIONS**
- Associated with phenomena
- Nonlinear interactions limit perturbation growth
- Eg, convective instabilities grow fast but are limited by availability of moisture etc
**LINEAR DESCRIPTION**
- May be valid at beginning stage only
- If linear models used, need to reflect nonlinear effects at given perturb. Amplitude
**BREEDING**
- Full nonlinear description
- Range of typical perturbation amplitudes is only free parameter

## DESCRIPTION OF NCEP ENSEMBLE FORECAST SYSTEMS

***OPERATIONAL***
- Global ensemble forecast system (based on MRF/GFS system)
- Limited Area Ensemble Forecast System (SREF, over NA)
***PLANNED***
- Seasonal Ensemble Forecast System (Planned, coupled model)
- 				***FOR EACH SYSTEM:***
- Configuration
- Initial perturbations
- Model perturbations
- Main users
- Applications
- Examples
- Discussion/Conclusion

## NCEP GLOBAL ENSEMBLE FORECAST SYSTEM

**CURRENT *****(APRIL 2003)***** SYSTEM**
- 10 members out to 16 days
- 2 *(4)* times daily
- T126 out to 3.5 *(7.5)* days
- Model error not yet represented
**PLANS**
***Initial perturbations***
- Rescale bred vectors via ETKF
- Perturb surface conditions
***Model errors***
- Push members apart
- Multiple physics (combinations)
- Change model to reflect uncertainties
***Post-processing***
- Multi-center ensembles
- Calibrate 1st & 2nd moment of pdf
- Multi-modal behavior?

## Slide 17

## BEST ESTIMATE OF FUTURE STATE

- RMS error
- Ensemble mean beats control
- Skill above climatology even in summer, out to 16 days
- Low resolution control beats hires control
- Ensemble spread
- Lower than ensemble mean error
- Due to lack of model perturbations

## No. Cases:     106      98        82        75        61       46

*No. Cases:     106      98        82        75        61       46*

## Slide 20

## Slide 21

## Slide 22

## Slide 23

## Slide 24

## Slide 25

## Slide 26

## Slide 27

## Slide 28

## Slide 29

## NCEP SHORT-RANGE ENSEMBLE FORECAST SYSTEM (SREF)

**OPERATIONAL SYSTEM**
- 10 Members out to 63 hrs
- 2 Models used:ETA & RSM
- 09 & 21 UTC initialization
- NA domain
- 48 km resolution
- Bred initial perturbations
- Products (on web):
- Ens. Mean & spread
- Spaghetti
- Probabilities
- Aviation specific
- Ongoing training

***PLANS***
*5 more members*
*More model diversity*
*4 cycles per day (3&15 UTC)*
*32 km resolution*
*New products*
*Aviation specific*
*AWIPS*
*Transition to WRF*

## PERTURBATION VS. ERROR CORRELATION ANALYSIS (PECA)

**METHOD****:** Compute correlation between ens perturbtns and error in control fcst for
- Individual members
- Optimal combination of members
- Each ensemble
- Various areas, all lead time
**EVALUATION:** Large correlation indicates ens captures error in control forecast
- Caveat – errors defined by analysis
**RESULTS:**
**Canadian** best on large scales
- Benefit of model diversity?
**ECMWF** gains most from combinations
- Benefit of orthogonalization?
**NCEP** best on small scale, short term
- Benefit of breeding (best estimate initial error)?
- PECA increases with lead time
- Lyapunov convergence
- Nonlilnear saturation
- Higher values on small scales