# Presentation: 624436

## Initial Perturbations Experiments for NCEP EPSMozheng Wei*, Zoltan Toth,Dick Wobus**, Yuejian Zhu NCEP/EMC, MD*UCAR at NCEP/EMC;    **SAIC at NCEP/EMC in collaboration with:   Craig Bishop*** and Xuguang Wang**** *** Naval Research Lab; **** CIRES/CDC Dec.  8, 2004  First Thorpex International ScienceSymposium, Montreal, Canada, Dec. 6-10, 2004

**Notes:**
Outline slide

## OUTLINE

**THE PROBLEM**
**DESCRIPTION OF VARIOUS SCHEMES**
**EXPERIMENTAL RESULTS**
**DISCUSSION**

## EXPOSITION OF PROBLEM

- TO REPRESENT ANALYSIS ERROR-RELATED FORECAST UNCERTAINTY:
- ESTIMATE analysis uncertainty
- Ideally, construct multidimensional PDF
- Prohibitive in practice, due to large dimensionality
- In practice, estimate analysis error covariance matrix
- Works in truncated space
- Case dependent estimate possible with ensemble schemes
- SAMPLE analysis uncertainty estimate
- Sample space of estimated analysis error
- Use random or directed sampling strategies
- INTEGRATE NWP model from sample of initial conditions
- Evaluate sample of integrations to assess forecast uncertainty
*EPS and DA systems must be consistent for best performance of both*

## EXISTING/PROPOSED APPROACHES

*** ******FIRST****** GENERATION INITIAL PERTURBATION GENERATION TECHNIQUES***

|  | PERTURBED OBSERVATIONS
(MSC, Canada) | BREEDING with Regional Rescaling
(NCEP, USA) | SINGULAR VECTORS with Total Energy
(ECMWF) |
| --- | --- | --- | --- |
| ESTIMATION | Realistic through sample, case dependent patterns & amplitudes | Fastest growing subspace, case dependent patterns | No explicit estimate, not flow dependent |
| SAMPLING | Random for all errors, incl. non-growing, potentially hurting short-range performance  | Random in subspace of fastest growing errors; Some dependence among perts. | Directed, dynamically fastest growing in future |
| CONSISTENCY BETWEEN ENS & DA SYSTEMS | Very good; quality of DA lagging behind 3DVAR? | Time-constant variance due to use of fixed mask | Poor, potentially hurting short-range performance |

## EXISTING/PROPOSED APPROACHES - 2

***SECOND****** GENERATION INITIAL PERTURBATION GENERATOIN TECHNIQUES***

|  | ETKF, perts  influenced by fcsts and observed data | ET/BREEDING with Analysis Error Variance Estimate from DA | SINGULAR VECTORS with Hessian norm |
| --- | --- | --- | --- |
| ESTIMATION | Fast growing subspace, case dependent patterns & amplitudes | Fastest growing subspace, case dependent patterns & amplitudes | Case dependent variance, climatologically fixed covariance  |
| SAMPLING | Orthogonal  in subspace of observations | Orthogonal in analysis covariance
norm | Directed, dynamically fastest growing in future |
| CONSISTENCY BETWEEN ENS & DA SYSTEMS | Very good; quality of DA lagging 4D-VAR? | Good; Error variance: DA=>ens; Error covariance: Ens=>DA | Climatologically consistent |

## COMPARISON OF DIFFERENT METHODS

**    ****GRADUAL CONVERGENCE OF METHODS?**
*ETKF with no observation perturbation = Breeding with orthogonalization and rescaling consistent with varying observational network*
- COMMON CONCEPT:
- Perturbations cycled dynamically through use of nonlinear integrations
- Bred Vectors (Toth & Kalnay 1993) = Nonlinear Lyapunov Vectors (Boffetta et al 1998)
*Evolved SVs constrained by analysis error covariance (Hessian SVs) ~ Bred perturbations*
- COMMON CONCEPT:
- With realistic initial constraint, SV dynamics ~ Lyapunov dynamics?
- Explore SVs in subspace of ensemble forecasts – Bishop,  etc

## MOTIVATION FOR EXPERIMENTS

***      ******TWO OJECTIVES *****for ensemble generation:**
- Best quality ensemble forecasts
- Primary objective, performance measure
- Ensemble as consistent with data assimilation system as possible
- Secondary objective, to facilitate use of ensemble info in DA
***CONSISTENCY****** ***can be achieved by:
- Development & use of ensemble-based DA system
- Through THORPEX project, NCEP is collaborating with 4-5 groups on this
- Coupling existing DA (3/4DVAR) with ensemble generation scheme
- Goal of present study
***INTEREST****** *****of study:**
- As long as ensemble-based DA cannot outperform other 3/4DVAR
- Modify and couple existing DA and ensemble systems
- Use cheap ensemble generation scheme, since full consistency is unreachable
- Simple initial perturbation scheme driven by analysis error variance from DA
- 3/4DVAR driven by flow dependent forecast error covariance from ensemble

## DESCRIPTION OF 4 METHODS TESTED

**BREEDING** with regional rescaling (Toth & Kalnay 1997)
- Simple scheme to dynamically recycle perturbations
- Variance constrained statistically by fixed analysis error estimate “mask”
***Limitations:*** No orthogonalization; fixed analysis variance estimate used
**ETKF **(Bishop et al. 2004, Wang & Bishop 2003) – used as
-             perturbation generator (not DA)
- Dynamical recycling as breeding, with orthogonalization in obs space
- Variance constrained by distribution & error variance of observations
- Constraint does not work well with only 10 ensemble members
- Built on ETKF DA assumptions => ***NOT consistent with 3/4DVAR***
**Ensemble Transform (ET)** (Bishop & Toth 1999)
- Dynamical recycling as breeding, with orthogonalization
- Variance constrained statistically by fixed analysis error estimate “mask”
- Constraint does not work well with only 10 ensemble members
**ET plus rescaling** = Breeding with orthogonalization, (Wei et al. 2004)
- As ET, except variance constrained statistically by fixed analysis error estimate

## EXPERIMENTS

- Time period
- Jan 15 – Feb 15 2003
- Data Assimilation
- NCEP SSI (3D-VAR)
- Model
- NCEP GFS model, T126L28
- Ensemble
- 2x5 or 10 members, no model perturbations
- Evaluation
- 7 measures, need to add probabilistic forecast performance

## Initial energy spread,      Rescaling factor distribution   

- ET

- ETKF

- Breeding

- ET+rescaling

## Amp Factor

- Amp Factor

- Effective Dim

- Correlation

## PECA

- Variance

## AC

- AC

- RMS error

## SUMMARY of RESULTS

**RMSE, PAC of ensemble mean forecast – ***Most important*
- ET+Rescaling and Breeding are best, ET worse, ETKF worst
**Perts and Fcst error correlation** (PECA) – *Important for DA*
- ET+Rescaling best, Breeding second
**Explained variance** (scatterplots) – *Important for DA*
- ET best
**Variance distribution** (climatological, geographically)
- Breeding, ET+Rescaling reasonable
**Growth rate**
- ET+Rescaling best? (not all runs had same initial variance...)
**Effective degrees of freedom** out of 5 members
- Minimal effect of orthogonalization
- Breeding (no orthogonalization) 	=4.6
- ET (built-in orthogonalization)	=4.7
**Time consistency** of perturbations (PAC between fcst vs. analysis perts)
- Important for hydrologic, ocean wave, etc ensemble forcing applications
- Excellent for all schemes, ET highest (0.999, breeding “lowest”, 0.988)
- New and very promising result for ET & ETKF
***OVERALL hits out of ******	******7***
- ET+Rescaling	4
- ET		3
- Breeding		2

## DISCUSSION

- All tests in context of 5-10 perturbations
- Will test with 80 members
- Plan to experimentally exchange members with NRL
- Will have total of 160 members
- 4-dim time-dependent estimate of analysis error variance
- Need to develop procedure to derive from SSI 3DVAR
- ET+Rescaling looks promising
- Orthogonalization appears to help breeding
- Cheap procedure, also used in targeting
- If ensemble-based DA can not beat 3/4DVAR
- Initial ens cloud need to be repositioned to center on 3/4DVAR analysis
- No need for sophisticated ens-based DA algorithm for generating initial perts?
-                            Good EPS                    Good DA