# Presentation: 892871

## Research activities of the local ensemble transform Kalman filter (LETKF) at JMA

- Takemasa Miyoshi
- Numerical Prediction Division, JMA

- 2007/10/8

## Role of data assimilation

- Major Methodologies in Atmos & Oceanic Sciences

**Data Assimilation**

- Observing scientists get dynamically consistent 4-D analysis

- Modeling scientists get information of real nature

- Initialize model predictions; essential in NWP

**Synergy between two major fields helps scientific advancement**

## Data assimilation in NWP

- time

- OBS

- OBS

_*True state (unknown)*_

*FCST-ANALYSIS Cycle* accumulates past observations

- Numerical Weather Prediction

## Data Assimilation = Analysis

*True state (unknown)*

- OBS

- OBS Error

- FCST

- ANL

- ANL Error

- FCST Error

- FCST

- The error in the initial condition (ANL) grows in a chaotic system

## Probabilistic view

- T=t0

- ANL w/ errors

- T=t1

## A schematic of EnKF

- T=t0

- T=t1

- Generate ensemble members best representing the analysis errors

- T=t2

- EnKF = ensemble fcst. + ensemble update

- An initial condition with errors

## EnKF - summary

- EnKF considers flow-dependent error structures, or the “errors of the day”
- “advanced” data assimilation method
- 4D-Var is also an “advanced” method. How different?
- EnKF analyzes the analysis errors in addition to analysis itself
- “ideal” ensemble perturbations

## Research activities

- MISSION STATEMENT
- Develop a next-generation data assimilation system to improve operational NWP at JMA

_***Path to operations***_

_***Researches using the Earth Simulator LETKF system***_

-  Experimental 1.5-yr reanalysis (ALERA)
-  Collaborative work with observing scientists
-  LETKF with the Earth Simulator coupled atmos-ocean GCM

-  Develop and test LETKF (Hunt 2005; Hunt et al. 2007; Ott et al. 2004, Hunt et al. 2004) with the Earth Simulator
-  Develop LETKF with the JMA nonhydrostatic model
-  Develop LETKF with the JMA global model
-  Assess LETKF under the quasi-operational setup

## Outline

- Developments of LETKF toward operations
- Recent improvements
- Quasi-operational comparison with 4D-Var
- Probabilistic forecast skills
- Research projects with the Earth Simulator
- Experimental Ensemble Reanalysis: ALERA
- Collaboration with observing scientists

## Developments toward operations

## LETKF

- Two categories of the EnKF (Ensemble Kalman Filter)
- LETKF (Local Ensemble Transform Kalman Filter)
- is a kind of ensemble square root filter (SRF)
- is efficient with the parallel architecture

| Perturbed observation (PO) method | Square root filter (SRF) |
| --- | --- |
| Classical | Relatively new |
| Already in operations (Canadian EPS) | Not in operations yet |
| Additional sampling errors by PO | No such additional sampling errors |

## LETKF developments at JMA

- LETKF (Local Ensemble Transform Kalman Filter) has been applied to 3 models
- AFES (AGCM for the Earth Simulator)
- NHM (JMA nonhydrostatic model)
- GSM (JMA global spectral model)

- Miyoshi and Yamane, 2007: *Mon. Wea. Rev.*, in press.

- Miyoshi, Yamane, and Enomoto, 2007: *SOLA*, 45-48.

- Miyoshi and Aranami, 2006: *SOLA*, 128-131.

- Miyoshi and Sato, 2007: *SOLA*, 37-40.

## Quasi-operational Experimental System

- 4D-Var

- LETKF

- Ensemble forecast
- 9hr TL159/L40

- Deterministic forecast
- 9hr TL319/L40

- QC

- QC

- 4D-Var

## Recent improvements

- Assimilation of satellite radiances
- greatly improves the analysis accuracy
- Removing local patches
- solves the discontinuity problem near the Poles
- Efficient MPI parallel implementation
- solves the load imbalance problem
- accelerates by a factor of 3
- Adaptive satellite bias correction
- a new idea analogous to the variational bias correction
- showing great positive impact

- Miyoshi and Sato, 2007: *SOLA*, 37-40.

- Miyoshi et al., 2007: *SOLA*, 89-92.

- about 30% faster than operational 4D-Var with similar settings

## Impact by satellite radiances

- Blue: w/o satellite radiances
- Red: w/ satellite radiances

- RMSE and bias against radiosondes

- 20 members

## Typhoon track ensemble prediction

- Typhoon #13, 2004

- Bred Vectors w/ 4D-Var
- (Currently in operations)

## Typhoon track ensemble prediction

- Typhoon #13, 2004

- SV w/ 4D-Var
- (Next operational system)

## Typhoon track ensemble prediction

- Typhoon #13, 2004

- LETKF (Developing)

## This was a very difficult typhoon case

- LETKF

- Operational Systems as of Aug 2004

- Best track

## Statistical typhoon track errors

- Typhoons in August 2004

## Satellite radiance bias correction

- Observation      has a bias

## Adaptive bias correction

- Coefficients would change partly due to the deterioration of sensors

- Allow temporal variation of the coefficients
- using data assimilation

- _Variational bias correction_ (e.g., Dee 2003; Sato 2007)

- Find the minimizer  of the cost function J through the variational procedure

## Adaptive bias correction with LETKF

- Analytical solution of the variational problem: minimizer (x, )

- 1. Solve the LETKF data assimilation problem first

- _Adaptive bias correction with LETKF_

- difference

- 2. Solve the equation for  explicitly

- This coincides with the variational BC

## Time series of bias coefficients

- AMSU-A 4ch (sensitive to middle-lower troposphere) indicates significant drift from those estimated by
- 4D-Var

- AMSU-A 6ch (sensitive to upper troposhere) and other sensors/channels indicate no significant drift

## Impact by adaptive bias correction

- 24hr temperature forecast error improvements relative to 4D-Var

- Apply Adaptive BC

- RED: LETKF is better
- BLUE: 4D-Var is better

- NH

- SH

- Tropics

## Bias reduction

- The improvements would be due to the bias reduction

- Global

- NH

- SH

- Tropics

- Apply Adaptive BC

- T850 forecast bias against initial condition

- Red: LETKF
- Blue: 4D-Var

## Look at other variables

- Almost everything in the SH
- Temperature below 800hPa
- Temperature above 700 hPa except SH
- Wind except SH

- T

- Z

- U

- Daily variations of 24-hr forecast error improvements relative to 4D-Var
- Horizontal axis: Date
- Red: LETKF is better
- Blue: 4D-Var is better

- Global

- NH

- SH

- Tropics

## With adaptive satellite bias correction

- T

- Z

- U

- Daily variations of 24-hr forecast error improvements relative to 4D-Var
- Horizontal axis: Date
- Red: LETKF is better
- Blue: 4D-Var is better

- Global

- NH

- SH

- Tropics

## Improvement (%) relative to 4D-Var

- Apply adaptive bias correction

## Improvement (%) relative to 4D-Var

- August 2004

- December 2005

- LETKF is advantageous in the summer hemisphere

## Comparison with 3D-Var

- NH

- SH

- Tropics

- AC

- Initial conditions

- Radiosondes

- RMSE

- BIAS

- RMSE

- BIAS

- Red: LETKF
- Blue: 3D-Var

- Z500

## Computational time

| LETKF | 4D-Var |
| --- | --- |
| 11 min x 60 nodes | 17 min x 60 nodes |
| 5 min for LETKF
6 min for 9-hr ensemble forecasts |  |
| TL319/L60/M50
 | Inner: T159/L60
Outer: TL959/L60 |

- Estimated for a proposed next generation operational condition

- Computation of LETKF is reasonably fast, good for the operational use.

- 6 min (measured) x 8 nodes for LETKF with TL159/L40/M50

## Probabilistic forecast skills

- High T > TCLM + 2K

**LETKF is advantageous up to 3-day forecasts**

## Probabilistic forecast skills

**LETKF is more sensitively affected by the negative temperature model bias**

- Low T < TCLM - 2K

## Economic value

- High T > TCLM + 2K

## Economic value

- Low T < TCLM - 2K

## Summary

- LETKF indicated identical performance in the NH to the operational 4D-Var
- However, LETKF is clearly worse than 4D-Var in the SH
- Larger bias in LETKF; why?
- LETKF is advantageous in Typhoon prediction
- Probabilistic forecast skill is generally improved
- Still bias problem

## Researches with the Earth Simulator

## Ensemble Reanalysis: ALERA

- T159/L48 AFES (AGCM for the ES)

- LETKF w/ 40 members

- Reanalysis from May 2005 through February 2007

- ALERA
- (AFES-LETKF Experimental Ensemble Reanalysis)

- Assimilate real observations used in the JMA operational global analysis, except for satellite radiances

## Stable performance

- Very Stable!

- System change: vertical localization of ps obs.

- (compared to NCEP/NCAR)

## Snapshot (SLP)

- ALERA

- NNR

- ALERA analysis is almost identical to NNR.

## Zonal mean winds in JJA

- ALERA

- NNR

## Comparison with observations

- T

- T

- AIRS retrievals

- Radiosondes

## Compared with radiosondes

- U

- Z

- Radiosondes

- Radiosondes

## 予報スコア

- Adapted from Miyoshi and Yamane (2007)

## ALERA dataset

- ALERA
- (AFES-LETKF Experimental Ensemble Reanalysis)

- data are now available online for free!!

- Available ‘AS-IS’ for free ONLY for research purposes
- Any feedback is greatly appreciated.

- Contents
- Ensemble reanalysis dataset for over 1.5 years since May 1, 2005
- 40 ensemble members
- ensemble mean
- ensemble spread

## Analyzing the analysis errors

- EnKF provides not only analysis itself but also analysis errors (or uncertainties of the analysis)
- What is dynamical meaning of the analysis errors?

- GOES-9 Image

- Courtesy of T. Enomoto

## QBO and ensemble spread

- Courtesy of T. Enomoto

- Large spread at the initial stage of phase change

## Stratospheric sudden warming

- NCEP/NCAR

- ALERA

- SPREAD

- Courtesy of T. Enomoto

## Large spread in tropical lower wind

- Courtesy of T. Enomoto

## Tropical lower wind and the spread

- SPREAD

- Courtesy of T. Enomoto

## Courtesy of T. Enomoto 

- Courtesy of T. Enomoto

## Collaboration with observing scientists

- There was an intensive observing project in the tropical western Pacific (a.k.a. PALAU-2005); the dropsonde obs during June 12-17, 2005 have been assimilated with the AFES-LETKF system.

- Moteki et al., 2007: *SOLA*, accepted.

- Satellite image and dropsonde locations

## Collaboration with observing scientists

- Moteki et al., 2007: *SOLA*, accepted.

_***Impact by dropsonde observations***_

- Ensemble spreads of the ensemble analysis by LETKF

- NO OBS

- DROPSONDES

## Collaboration with observing scientists

_***Propagation of observing signals***_

- Moteki et al., 2007: *SOLA*, accepted.

## Summary

## Development plans

- Quasi-operational comparison in winter months
- Try to replace SV EPS with LETKF first
- Developments for high-resolution deterministic forecasts
- TL959 (about 20-km resolution) global system
- Incremental method (analogous to incremental 4D-Var)
- Improve skills in the SH (bias problem!)

## Research plans with the ES

- ALERA-2
- 5-yr reanalysis (21st century reanalysis)
- AFES-MATSIRO (SiB) (atmos-land coupled)
- More diagnostics (OLR, Precipitation, land variables, etc.; any requests?)
- CFES-LETKF
- LETKF with coupled atmos-land-ocean model
- More observing projects (e.g., PALAU-2008)

## Other research activities with LETKF

- Collaboration with chemical-transport modeling scientists at the Meteorological Research Institute
- LETKF has begun to work recently

## Collaborators

_*AFES-LETKF*_
- Prof. Shozo Yamane (Chiba Institute of Science and FRCGC/JAMSTEC, AFES)
- Dr. Takeshi Enomoto (ESC/JAMSTEC, AFES)
- Dr. Qoosaku Moteki (IORGC/JAMSTEC, Observing scientist)
_*NHM-LETKF*_
- Kohei Aranami (NPD/JMA, NHM)
- Dr. Hiromu Seko (MRI/JMA, DA with NHM)
_*GSM-LETKF*_
- Yoshiaki Sato (NPD/JMA, staying at NCEP)
- Takashi Kadowaki (NPD/JMA, 4D-Var)
- Ryouta Sakai (NPD/JMA, EPS)
- Munehiko Yamaguchi (NPD/JMA, Typhoon EPS)
_*Chemical Transport*_
- Dr. Thomas Sekiyama (MRI/JMA, Chemical model)

## Related publications

- AFES-LETKF
-      Miyoshi and Yamane, 2007: *MWR*, in press. (AFES-LETKF system)
-      Miyoshi et al., 2007: *SOLA*, **3**, 89-92. (LETKF without local patches)
-      Miyoshi et al., 2007: *SOLA*, **3**, 45-48. (ALERA)
-      Moteki et al., 2007: *SOLA*, accepted. (Observing project)
- NHM-LETKF
-      Miyoshi and Aranami, 2006: *SOLA*, **2**, 128-131. (perfect model experiments)
- GSM-LETKF
-      Miyoshi and Sato, 2007: *SOLA*, **3**, 37-40. (satellite radiance assimilation)

## Thank you

- for your attention!

## A core concept of EnKF

- Data Assimilation

- Ensemble Forecasting

- ANL Error

- FCST error

- This cycle process = EnKF
- Analyze w/ the flow-dependent forecast error, ensemble forecast w/ initial ensemble reflecting the analysis error

*Complementary relationship* between data assimilation and ensemble forecasting

## Difference between EnKF and 3D-Var

## An example of EnKF analysis accuracy

- EnKF is advantageous to traditional data assimilation methods including 3D-Var, currently in operations at several NWP centers.

- 3DVAR (31m)

- LEKF (8m)

- Serial EnSRF (5m)

- 30 ensemble members

- Many centers (ECMWF, JMA, UK MetOffice, MeteorFrance, Canada, etc.) switched to 4D-Var which also considers flow-dependent error structures.

## EnKF vs. 4D-Var

|  | EnKF | 4D-Var |
| --- | --- | --- |
| “advanced” method? | Y | Y |
| Simple to code? | Y | N (e.g., Minimizer)  |
| Adjoint model? | N | Y |
| Observation operator | Only forward
(e.g., TC center) | Adjoint required |
| Asynchronous obs? | Y (4D-EnKF) | Y (intrinsic) |
| Initialization after analysis? | N | Y |
| Analysis errors? | Y (ensemble ptb) | N |
| Limitation | ensemble size  | Assim. window |
|  | EnKF with infinite ensemble size and 4D-Var with infinite window are equivalent. |  |

## KF and EnKF

| Kalman Filter | Ensemble Kalman Filter |
| --- | --- |
|  |  |

- Solve for the ensemble mean

_*Forecast equations*_

_*Ensemble forecasts*_

- Approximated by

_*Kalman gain*_

_*Analysis equations*_

- Ensemble perturbations

- [pxp] matrix inverse

## LETKF algorithm (Hunt, 2005, et al., 2007)

- In the space spanned by

- Eigenvalue decomposition:

_*Analysis equations*_

_*LETKF analysis*_

- Ensemble analysis increments

## Ensemble size 20  50

- _RMSE and bias against radiosondes_

- Blue: Operational 4D-Var
- Red: 20-member LETKF
- Green: 50-member LETKF

- w/ satellite radiances

- Generally 4D-Var > LETKF
- Exception: mid-upper tropospheric temperature in the SH

- 50 members > 20 members

## 500 hPa height forecast verifications

- Red: LETKF  Blue: 4D-Var

- Global

- NH

- SH

- Tropics

## With adaptive satellite bias correction

- Red: LETKF  Blue: 4D-Var

- Global

- NH

- SH

- Tropics

## QBO (time series of tropical wind)

- ALERA

- NNR

## Stratospheric sudden warming

- ALERA

- NNR

## Stratospheric sudden warming