Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution
Takemasa Miyoshi* † and Shozo Yamane***Numerical Prediction Division, Japan Meteorological Agency**Chiba Institute of Science

In August 2005, we began to develop the local ensemble transform Kalman filter (LETKF, Hunt2005) with AFES (AGCM for the Earth Simulator, Ohfuchi et al. 2004) at a T159 horizontal and 48-levelvertical resolution in collaboration among the Numerical Prediction Division, Japan MeteorologicalAgency (NPD/JMA), the Chiba Institute of Science, and the Earth Simulator Center (ESC). We developed
the LETKF system based on the system constructed by Miyoshi (2005) at the University of Maryland
where originally LETKF has been invented.
A main limitation of ensemble Kalman filter (EnKF) experiments published thus far is the lowerresolutions than the currently operational data assimilation or ensemble prediction systems (EPS). The
experimental resolutions are at most as large as T62, whereas operational EPS have at least as large as aT106 resolution. A higher resolution model uses more precise physical processes and resolves smallerscale phenomena, possibly introducing larger substantial degrees of freedom of the dynamical error
structures. This causes the increase of required ensemble size for stable EnKF, an important disadvantage.
This research tackles the limitation by applying LETKF to the T159 AFES model, corresponding grid sizeof 480x240x48, a similar resolution to operational systems.
Generally EnKF has two parameters: the covariance inflation parameter and covariance localization
scale parameter. The former parameter is objectively estimated as in Miyoshi (2005) following asuggestion by Kalnay (pers. comm.). LETKF requires two types of localization parameters: the local
patch size and observational localization scale. In LETKF, it is better that we choose local patch size
sufficiently larger than the observational localization scale, although larger local patch size requires morecomputations. We assume horizontal isotropy, but vertically we define different localization parameters.
We choose 11x11x5 local patch and 2.0-grid horizontal and 1.0-grid vertical observational localizations in
the following experiments.
We follow observing systems simulation experiment (OSSE) methodology in a perfect model
scenario, where we generate the true nature run by a long-term integration of the same T159/L48 AFES
model and sample observations from it. We take observations every 6 hours in the time domain and in thespatial domain regularly one at every 5x5x4 grid points, just 1% of the entire grid points. The observed
variables are the zonal and meridional wind components, temperature, specific humidity, and surfacepressure with error standard deviations of 1.0 m/s, 1.0 K, 0.1 g/kg, and 1.0 hPa, respectively. The initialensemble members to initiate the data assimilation cycle are randomly chosen from a nature run in asimilar season, the initial ensemble mean is an analogue of the climatological mean.
Fig. 1 shows time series of the analysis root mean square errors (RMSE) of the surface pressure for
30 days. It is noted that even with 10 members, LETKF seems stable, at least it is not diverging.
Increasing ensemble size, we see LETKF be more stable with lower error levels, about half as large as theobservational error level. Fig. 2 shows the sensitivity to the ensemble size. At the first assimilation step,
we see linear decrease of the analysis errors up to about 80 members. Increasing the ensemble size
beyond 320 does not show significant change of the analysis error levels, which is consistent with the factthat the substantial degrees of freedom in the local patch are around 400. After 10-day data assimilation
cycles, the error decreasing trend is slower than linear, the difference between 40 and 80 members
becomes smaller.
Table 1 shows timing results of one-step LETKF computations without forecast computations. If
we use as many computational nodes as the ensemble size and they are less than 80, it takes just less than
4 minutes on the Earth Simulator, peak performance of 64 GFlops per node. The parallel efficiency
increases with ensemble size, about 99.99% parallelization ratio with more than 80 members.
†Corresponding author address: Numerical Prediction Division, Japan Meteorological Agency, 1-3-4Ote-machi, Chiyoda-ku, Tokyo 100-8122, Japan. E-mail: miyoshi@naps.kishou.go.jp

In summary, we developed and tested LETKF with the T159/L48 AFES model and found that evenwith the resolution as high as operational EPS, ensemble size of the order of less than 100 shows quite
stable and good filter performance. The computational time is also applicable for operations, we are now
at a stage of developing realistic observational operators and assimilating real observations.
AFES-LETKF (T159L48) ONE-STEP ANALYSIS ERRORS in TOTAL ENERGYANALYSIS ERRORS in TOTAL ENERGY

1.00E+22FIRST STEP
10 DAYS LATER

1.00E+21

1.00E+20

1.00E+1910204080

160

320

500

1000

ENSEMBLE SIZEFigure 1. Time series of the analysis errors of LETKF for 30days with 10 (black long-short-dashed line), 20 (red brokenline), 40 (green dashed line), and 80 (blue solid line)
ensemble members. The errors are measured by the RMSE ofthe surface pressure.

Ensemblesize
1020
4080
160
320
500
1000

5nodes
382
714
1389

Figure 2. Analysis errors in the total energy norm at one-stepanalyses of LETKF with changing the ensemble size. Solidand dashed lines show the analyses at the first-step and 10
days later from the first-step, respectively.

Table 1 Timing (sec.) of LETKF on the Earth Simulator
10204080160
nodes
nodes
nodes
nodes
nodes
195
115
67
47
360
197
107
66
708
360
189
112
1583
824
413
220
1205
626
336
1139

200
nodes

2270
10595

240
nodes

1924

Acknowledgements
We thank members of the chaos/weather group of the University of Maryland, especially Prof.
Eugenia Kalnay for fruitful discussions. We also thank Drs. Wataru Ohfuchi and Takeshi Enomoto of ESC,and Yoshiaki Takeuchi and Ko Koizumi of NPD/JMA for kind understanding and support for this project.
Most computations of this research have been performed on the Earth Simulator.
References
Hunt, B. R., 2005: Efficient Data Assimilation for Spatiotemporal Chaos: a Local Ensemble Transform
Kalman Filter. arXiv:physics/0511236v1, 25pp.
Miyoshi, T., 2005: Ensemble Kalman filter experiments with a primitive-equation global model. Ph.D.
dissertation, University of Maryland, 197pp.
Ohfuchi, W., H. Nakamura, M. K. Yoshioka, T. Enomoto, K. Takaya, X. Peng, S. Yamane, T. Nishimura,
Y. Kurihara, and K. Ninomiya, 2004: 10-km mesh meso-scale resolving simulations of the global
atmosphere on the Earth Simulator: Preliminary outcomes of AFES (AGCM for the Earth Simulator).
J. Earth Simulator, 1, 8–34.