**SUMMARY OF TELECONFERENCE DISCUSSION**

**December 18, 2008, 11:00am -- 1:00 pm EST**

**Present:** J. Whitaker and T. Hamill and X. Wang(NOAA ESRL), C. Bishop
and Dan Hodyss (NRL), M. Zupanski (CIRA/CSU), I. Szunyogh and E.
Kalnay(UMD), Z. Toth, M. Wei and M. Pena (NCEP/EMC)

**Agenda:**

1\) Review salient issues from Buenos Aires meeting (30-45 mins)\
2)   Review of new developments/results from each group since last
meeting

(10-15 mins max each)\
3)   Discussion on coordination among groups

Eugenia Kalnay:

Introduced the Workshop on 4D-Var and ensemble Kalman Filter
Inter-comparisons which was held in Buenos Aires, Argentina from 10-13
November 2008. The outcome from this workshop demonstrated the
comparative performance of EnKF method compared with variational method
which is widely used at NWP centers. Most of the time was focused on the
inter-comparison of 4D-Var and ENKF systems at Canadian system presented
by Mark Buehner. Both 4D-Var and EnKF have been operational at CMC since
2005, both use the same forecast model and similar set of observations.
Thus the two systems at CMC provide a quite clean comparison between the
two methods. The conclusion is that 4D-Var and EnKF analyses have
comparable quality, while 4D-Var with flow-dependent EnKF covariances
yields a gain of about 10 hours at day 5 in southern extra-tropics. Work
is still on the way to complete EnKF experiment using incremental
approach to produce high-resolution deterministic analysis. The results
from JMA show EnKF has worse results in SH, but better in NH due to the
model bias problem. Overall, participants were excited to see how close
the results produced by the two methods have become.

Her work at University of Maryland includes the work on CO2 in DA with
SPEED model. The goal is to compare LETKF with the inverse method.

Craig Bishop:

Sent slides showing the comparison of localization in radiance space and
model space. Explained why adding satellite radiances improve more in
variational DA system than EnKF system. The results show the radiance
space localization is inferior to model space localization.

Jeff Whitaker:

Spent more time on software engineering, as a result EnKF code is
one-order of magnitude faster than last year. He is focusing more on
bias correction and applying the EnKF system on hurricane study. Future
work will include the development of hybrid approach. Pointed out that
there are differences between the ways of ensembles being used at the
CMC and NCEP. The CMC method take full advantage of flow-dependent
information from the ensembles.

Later on Dec. 30, 2008, Jeff sent slides that show great improvement of
EnKF over GSI and the speed of codes is much faster than before. See the
slides (on the same website) for more details.

Milija Zupanski:

Introduced his recent work, focused more on much higher resolution of DA
targeting for microphysics processes at 9-13km. At this resolution, EnKF
produces much better picture than GSI. In his method, the LETKF is used
in sub-domain smoothing.

Istvan Szunyogh:

His work includes working on chemical DA with OZ, but pointed out there
are challenges in localization for OZ. It will be interesting to study
the vertical localization. Another area is EnKF in coupled global system
that involves both atmosphere and ocean. He is also continuing his work
on predictability. The effort is on using ensemble to represent the
uncertainty in the system.

Zoltan Toth:

Summarized the tele-conference and the activities in the past year. He
congratulated Tom Hamill who will take over as the manager of NOAA
Thorpex activity from next year. Suggested the next tele-conference in
March 2009.

Tom Hamill expressed his thanks to Zoltan Toth for his work during the
past years in organizing NOAA Thorpex DA activities.

Mozheng Wei's slides on analysis differences and error variation from
different NWP centers was postponed to the next time due to the time
limit.

**SUMMARY OF TELECONFERENCE DISCUSSION**

**April 1, 2008, 1:00pm -- 3:00 pm EST**

**Present:** J. Whitaker and T. Hamill (CDC), C. Bishop (NRL), M.
Zupanski (CIRA/CSU), I. Szunyogh and E. Kalnay(UMD), Y. Song, Z. Toth,
M. Wei and Steve Lord (NCEP/EMC), E. Kalnay (UMD). Gregory Hakim (Uni.
of Washington)

**Agenda:**

> 1\) Introduction of DA activity and plan by Steve Lord.
>
> 2). Review of progress by each group (5-10 minute each group with
> slides
>
> if possible)
>
> 2). Plans for the next 3-6 months by each group
>
> 3). Longer term issues-expected benefits & computational costs of
> ens-DA
>
> and hybrid variational approach.

Steve Lord:

Steve introduced NCEP/EMC DA activity and plan. Various issues have been
raised and discussed. EMC has been developing next version of GSI called
FOTO (First-Order Time extrapolation to the Observation). FOTO is a
relatively efficient technique for taking into account temporal
distribution of the observations within a 3D-VAR data assimilation
system. The experimental results demonstrate the positive impact over
the current operational GSI. FOTO is expected to be implemented in May
2008. EMC is continuing cooperation with NASA DA Office in development
of tangent linear model of GSI towards an eventual full version of
4D-Var depending on its final performance. He also talked about the
difficulty of contributing more computing resources to Thorpex funded
ensemble DA activities before the arrival of the NECP new supercomputers
which is expected to be in June 2008.

Milija Zupanski:

Sent slides showing the application of localized MLEF plus WRF regional
model to Hurricane Katrina. In this method, he used error covariance
localization based on a modified local domains approach. The domain
studied is divided into 9 local domains with each containing **23x25**
horizontal grid points. The observation weight matrix is a linear
interpolation from neighboring local domains. Positive impact has been
found in application to Hurricane Katrina case study.

Jeff Whitaker:

New slides showed the comparison results between GSI, LETKF and EnSRF.
Code has been ported to NCEP system and optimized. The comparison can be
made with operational GSI and focus more on forecasts of hydrologic
variables like humidity, precipitation and probabilistic forecast
verification. The results are for period from May 15 to June 15, 2007.
The model resolution is T126L64 with 64 ensembles. There is almost no
difference between EDA and GSI in terms of forecast RMS errors verified
against own analyses. For RMS error of precipitable water, EDA forecasts
are significantly more skillful than GSI. EDA is much better in tropics
and significantly better at short leads time over CONUS. In terms of
ensemble forecasts of T850, EnKF skill is slightly larger at the
shortest lead time but quickly loses to ET. ET spreads grow more quickly
than EnKF. Overall, computational cost for EnKF has been reduced.

Eugenia Kalnay:

Sent some slides and a short paper describing a new method for
accelerating the spin-up of ensemble Kalman filter so that it even
converges faster than 4D-Var. It uses a "running in place" algorithm
which is made possible by the use of "no-cost" ensemble Kalman Smoother
(EnKS) proposed by Kalnay et al. (2007) and tested by Yang et al.
(2008). During the process, the observations from the past were used
more than once by running ensemble Kalman Smoother.

Mozheng Wei:

Showed slides and updated the EMC progress in estimating analysis error
variances from GSI. The pre-conditioned conjugate gradient methods in
GSI is used to deliver two sets of gradients vectors, one is gradient
with respect to **x** (**g**) and the other is with respect to the
transformed vector **z** (**h**) (subroutine pcgsoi.f90 was modified).
These two sets form a bi-orthogonal system. The method consists of using
sub-sets of these vectors to estimate the Hessian matrix. It capitalizes
on relationship between conjugate gradient and the Lanczos algorithms
for solving large scale eigenvalue problem. This method was first
proposed by Fisher and Courtier (1995). The experiment has shown that
with a small number of conventional obs data, the reductions of error
variances in observation regions are identified with 30 dominant
eigenvectors. It is still a challenge to convert the variances of stream
function and velocity potential into the variances of **u** and **v**
which will be more useful in ensemble. Craig Bishop's suggestion of
using 30 eigenvectors will be investigated. Another difficulty is to get
accurate case-dependent background variances in order to get the
accurate analysis error variances. One possible solution is to use the
6-hour ensemble forecasts to construct background error variances.
Another different direction of this work was also mentioned, that is to
use the analysis data from different NWP centers.

Craig Bishop:

Sent some slides and two submitted manuscripts about his new adaptive
error covariance localization for 4-dimensional ensemble data
assimilation. This technique is called ECO-RAP (Ensemble COrrelations
RAised to a Power). ECO-RAP is a new flow-adaptive localization method
for ensemble DA. It raises ensemble correlations to a power to
selectively reduce spurious correlations, and it adapts to changes in
the propagation and scale characteristics of errors. The results show
that the new method outperforms non-adaptive localization method when
true errors are propagating or the error correlation length scale is
varying.

Zoltan Toth:

Summarized the activities and progress so far and planned to have the
next meeting in June or July 2008. Groups discussed the future of Ens-DA
activities. Craig Bishop suggested looking at observation data which
need to be improved, especially satellite data bias correction needs to
be improved. Mozheng Wei suggested to use a new GSI benchmark based on
the new implementation of FOTO in May 2008. This view was shared by Jeff
Whitaker and Tom Hamill.

**SUMMARY OF TELECONFERENCE DISCUSSION**

**October 4, 2007, 1:00pm -- 3:00 pm EST**

**Present:** J. Whitaker, T. Hamill and X. Wang (CDC), C. Bishop and Dan
Hodyss (NRL), M. Zupanski (CIRA/CSU), Jeff Anderson (NCAR) I. Szunyogh
and E. Kalnay(UMD), Y. Song, Z. Toth and M. Wei (NCEP/EMC) , E.
Kostelich (Arizona State Uni.). Fuqing Zhang (Texas A&M Uni.). E. Kalnay
(UMD) and Joe Tribbia (NCAR) unable to attend.

**Agenda:**

> 1). Review of progress by each group (5-10 minute each group with
> slides
>
> if possible)
>
> 2). Plans for the next 3-6 months by each group
>
> 3). Longer term issues-expected benefits & computational costs of
> ens-DA
>
> and hybrid variational approach.

Craig Bishop:

Sent many slides and showed the results from his new adaptive error
covariance localization for 4-dimensional ensemble data assimilation.
This technique is called ECO-RAP (Ensemble COrrelations RAised to a
Power). ECO-RAP is a new flow-adaptive localization method for ensemble
DA. It raises ensemble correlations to a power to selectively reduce
spurious correlations, and it adapts to changes in the propagation and
scale characterisitics of errors.

2^nd^ part of the slides showed how to generate very large ensemble from
smaller scale turbulence ensembles. The method together with ECO=RAP was
tested in T119L30 model using LETKF, and produced promising results. In
the near future, he can help Jeff Whitaker and I. Szunyogh to implement
these techniques in their ensemble DA system..

Milija Zupanski:

Sent slides showing how to compute the minimization in MLEF with
non-differentiable observation operator based on 1-d Burgers model
simulating a shock-wave. Further slides showed some preliminary results
from MELF with WRF and real observations. When it was applied to
Hurricane Katrina, it showed the impact of observations comparing with
no-DA experiments.

He will continue to work on some computer issues and research on this
minimization method.

Jeff Whitaker:

New slides showed the results for May and June 2007 experiments and
compared with GSI in the same model resolution (T126L64) using 64
ensembles. With more observations added, the gap between the GSI and
LETKF is narrower than before. When AMSUA data are used, LETKF is
basically the same as GSI in N. Hem. in terms of O-F statistics. In the
Tropics, LETKF is slightly better for channels 14 and 15, but a little
worse for channels 1, 2 and 3. In the S. Hem., GSI is a little better
for channels 12, 13, 14 and 15.

When Radiosondes are used, GSI is better for temperature O-F in most
cases. In single observation experiments, it is shown that the
localization scheme in LETKF needs to be improved further in comparison
with the GSI.

In terms of computational cost, GSI (300secs) is much more efficient
than LETKF (3000secs) which runs faster than EnSRF (7000secs). The next
possible step is to work with Xuguang Wang testing hybrid method, and
with Criag Bishop and Milijia Zupanski to test their methods.

Istvan Szunyogh:

New slides were presented and showed some results from assimilation of
satellite radiances using cutoff-based data selection strategy. Bias
correction is done by augmenting the state with bias parameters in the
LETKF. The method was shown to improve the overall performance compared
with the other methods. Digital filter has been used. He has plan to
investigate the balance issues further in the future studies.

Fuqing Zhang:

Showed his previous slides that demonstrated the advantages of EnKF over
3D-WRF. One new slide in this conference showed the comparison between
GSI and EnKF in predicting hurricane Humberto (2007). The forecasts from
EnKF are clearly better than those from GSI analysis in predicting the
pressure and wind speed near-shore during hurricane Humberto. However,
one should be careful when evaluating the results. Running a mesoscale
forecast initialized with the coarser resolution global GSI may not
offer a fair comparison of the capabilities of the 2 DA schemes
compared.

Xuguang Wang:

Presented a hybrid ETKF-3DVAR data assimilation scheme for WRF. The
hybrid system has shown larger improvement over data sparse regions,
such as ocean. The flow-dependent ensemble covariance has shown the
largest impact on the regions with sparse observations. The forecasts
from the hybrid system with real observations also outperformed the
forecasts from WRF 3DVAR system which is still under development (not as
mature as GSI).

Eugenia Kalnay:

Sent 2 sets of slides about sparse computations within LETKF, and
studies about observation impact within the LETKF without adjoint. But
she was unable to join. Material will be presented at the next
tele-conference.

Jeff Anderson:

Pointed out that a global version of WRF is available for use. He
suggested to use different localization schemes for different places,
different types of observations and different model variables.

Mozheng Wei:

Updated the DA activity using ensemble at EMC. DA group at EMC is
planning to use ensemble information to improve the background
covariance matrix in GSI.

Zoltan Toth:

Summarized the activities and progress so far and planned to have the
next meeting in the first half of January 2008. Discussed future Ens-DA
activities.

**SUMMARY OF TELECONFERENCE DISCUSSION**

**May 17, 2007, 1:00pm -- 3:00 pm EST**

**Present:** J. Whitaker and T. Hamill (CDC), C. Bishop and Dan Hodyss
(NRL), M. Zupanski (CIRA/CSU), Jeff Anderson and Joe Tribbia (NCAR) I.
Szunyogh and E. Kalnay(UMD), Y. Song, Z. Toth and M. Wei (NCEP/EMC) , E.
Kostelich (Arizona State Uni.)

**Agenda:**

> 1). Review of progress by each group (5-10 minute each group with
> slides
>
> if possible)
>
> 2). Plans for the next 3-6 months by each group
>
> 3). Status of benchmark, discussion on comparison of results
>
> 4). Longer term issues-expected benefits & computational costs of
> ens-DA
>
> and hybrid variational approach.

Jeff Whitaker:

Reported the progress he has made. Modified the DA code for new GFS
model, and new GSI will be used for forward operator (GSI has been
implemented since May 1, 2007) in ensemble DA. New codes have been
ported and debugging has started at T126. Satellite data have not shown
much benefit for the ensemble DA analysis. They are looking at
flow-dependent error covariance and model errors. Increasing model
resolution is expected to reduce the role of model errors.

Eugenia Kalnay:

Sent some slides and presentations from recent meetings. The experiments
using Quasi-Geostrophic model showed that 40-member LETKF is better than
12-hour 4D-Var which is better than hybrid (3D/Var + 20 BVs), while
3D-Var and 24-hour 4D-Var are the worst and best respectively. Singular
vectors and bred vectors are compared with the errors of the day. It was
demonstrated that the errors of the day are most important for providing
background error covariance in LETKF and 3D-Var.

Jeff Anderson:

Reported the newly released MPI version of DART using WRF model. Ran
test with re-analysis observations. New inflation strategy, which is
called adaptive inflation, greatly improved the analysis results.

Milija Zupanski:

Still working with setting up computers. Was working on development of
new ensemble localization scheme that will take time into account.

Istvan Szunyogh:

Finished a final version of paper for Physica D that summarizes the
results based on LETKF without satellite radiances. Implemented LETKF on
NCEP model and produced some good results from using satellite data.
Gave support to Jeff Whitaker's experiments.

Craig Bishop:

Sent some new slides to show the results from his new localization
scheme where a SENCORP moderation function is constructed from
equivalent potential temperature correlation filed smoothing in both
horizontal and vertical using a Gaussian weighting function in Fourier
phase space was performed. He showed the impact of using these
moderation function in DA on forecast and analysis with NOGAPS
ensembles. It is going to be tested in NVADAS system at NRL first. In
the future it can be used in Whitaker's ens-DA system.

Mozheng Wei:

Introduced the DA activity at EMC. DA group at EMC is planning to use
ensemble information to improve the background covariance matrix in GSI.
The initial plan is to use the hybrid method which consists of both
static 3D-Var background and flow-dependent ensemble based error
covariance matrix. Ensemble group is hoping to get the estimate of
analysis error variance from GSI for ensemble use in initialization.

Zoltan Toth:

Planned the next meeting in August 2007. Future Ens-DA should work with
regional, ocean and global ensembles. Two-way comparison with future
4D-Var system. Proposed to create a new benchmark at T126L64 from GSI
with new observations from May 1, 2007 when GSI was implemented.

**SUMMARY OF TELECONFERENCE DISCUSSION**

**January 30, 2007, 11am -- 1:30 pm EST**

**Present:** J. Whitaker and T. Hamill (CDC), C. Bishop and Dan Hodyss
(NRL), M. Zupanski (CIRA/CSU), Joe Tribbia (NCAR)I. Szunyogh and E
Kalnay(UMD), Y. Song, Z. Toth and M. Wei (NCEP/EMC) , E. Kostelich
(Arizona State Uni.), Fuqing Zhang (Texas A&M), Gregory Hakim (Uni. of
Washington)

**Agenda:**

> 1). Review of progress by each group (5-10 minute each group with
> slides
>
> if possible)
>
> 2). Plans for the next 3-6 months by each group
>
> 3). Status of benchmark, discussion on comparison of results
>
> 4). Longer term issues-expected benefits & computational costs of
> ens-DA
>
> and hybrid variational approach.
>
> 5). Link wirh regional DA efforts (Grep Hakim and Fuqing Zhang joined
>
> discussion)

Istvan Szunyogh:

Finished a paper that summarizes the results based on LETKF without
satellite radiances. In the new code, bias correction can be done in
different ways. Results can be compared with different bias correction
schemes. The code has been tested by Jeff Whitaker to compare with EnSRF
in the same environment. The differences from these two algorithms are
not significant right now. More real observations are going to be
tested.

Eugenia Kalnay:

Suggested to try Milija Zupanski's MLEF idea in the system Jeff Whitaker
is testing. Reported different ways of turning inflation factors for
background error covariance. Experimented different ways of adaptive
tuning of observation errors.

Jeff Whitaker and Tom Hamill:

Sent the slides he showed at Jan 2007 AMS meeting. The results showed
that 6-12 hours forecast improvement at 100hPa is more than at 850hPa.
Improvement is largest in data sparse regions, such as SH or
stratosphere. A radiative transfer model was used, carried out bias
correction in radiance observations, e.g. scan-angle correction and
using linear regression model. The bias coefficients in Air-Mass bias
correction scheme were updated using global ETKF equations.

Craig Bishop

Based on the slides from last tele-conference, described more detail
about his method for moderating spurious ensemble correlations. The
method called Smoothed ENsemble Correlations Raised to a Power (SENCORP)
is proposed to provide flow adaptive moderations functions in comparison
with the conventional method used in current DA systems. It has shown
advantage over hierarchical ensemble filter which is subject to noise
when tested in a simplified system. The next step is to test it in NOGAP
system at Navy. A full description of this method is expected at later
stage.

Milija Zupanski:

Work on testing MLEF (maximum likelihood ensemble filter) is continuing
using real observations and satellite radiance data. A lot of attention
has been on localization using background error covariance information.
Plan to collaborate with experts in computer science department at CSU
and more research will be done. Also on agenda is testing a local
version of MLEF.

Fuqing Zhang:

Introduced his group's work on regional scale EnKF in comparison with
WRF 3D-Var. In his experiments, it was found that EnKF consistently
produced better results than WFR 3D-Var. The benefit from using ensemble
mean to initialize the forecast has been demonstrated. Pointed out that
multi-physics model also improves analysis results.

Greg Hakim:

Introduced the operational EnKF data assimilation system over North
America in University of Washington. Pointed out a lot of results from
their group available on web site. The Washington EnKF system is
focusing on North America. The boundary conditions are generated from
NCEP GFS forecasts. They found larger noise from the boundary. More
research work needs to be done to reduce the noise near boundary.

Zoltan Toth:

Planned to schedule the next tele-conference during March-April 2007.
Raised possibility of possible future comparison or implementation of
regional ensemble DA at NCEP. Other questions raised: Do we need to run
T254 benchmark? How to best tune observation errors and background error
inflation and localization? Exploit Milija's adaptive localization
technique.

**SUMMARY OF TELECONFERENCE DISCUSSION**

**October 19, 2006, 1 0am -- 1 pm EST**

**Present:** J. Whitaker and T. Hamill (CDC), C. Bishop (NRL), M.
Zupanski (CIRA/CSU), I. Szunyogh (UMD), Y. Song, Z. Toth and M. Wei
(NCEP/EMC) , E. Kostelich (Arizona State Uni.)

**Agenda:**

> 1). Review of progress by each group (with slides if possible)
>
> 2). Plans for the next 3-6 months by each group
>
> 3). Status of benchmark, discussion on unified verification of results

Jeff Whitaker and Tom Hamill:

Sent 3 slides prior to the meeting. One of the slides shows the
schematic of ensemble DA flow with 2 members. The DA system uses NCEP
SSI to compute observation operator which maps the model state to
observational points. Bias correction is carried out by inputting
ensemble mean to SSI. The DA system can be easily switched to LETKF from
EnSRF. Several inflation schemes have been built in including additive
inflation which is being used to produce the results in the figures. The
results without satellite data show that both ensemble DA systems are
much better than SSI in the SH where the observations are less. The
differences between LETKF and EnSRF are not significant. However, LETKF
is much faster, possibly due to the fact that LETKF computes Hx itself,
while EnSRF calls SSI. But it is also harder to do data thinning in
LETKF. When satellite data are included. Only results from LETKF are
shown. They are similar to SSI, except in SH where SSI is better.

Craig Bishop

Showed 18 slides about moderating spurious ensemble correlations
continuing their work on covariance localization for ensemble based
Kalman filter data assimilation system. A method of using Smoothed
ENsemble Correlations Raised to a Power (SENCORP) moderation functions
is proposed to provide flow adaptive moderations functions in comparison
with the conventional method used in current DA systems. A fixed
moderation function is multiplied in the convention method. This method
can also be used to simulate model errors. Again the experiments are
carried out on an extremely simple system. The results are positive.
Jeff Whitaker expressed interest in testing this method in his ensemble
Kalman filter DA system. A full description of this method is expected
at later stage.

Istvan Szunyogh:

Real observations are used for LETKF formulation which uses ETKF
solution for a small patch at each grid point. The algorithm is more
efficient than EnSRF algorithm during the experiments with and without
satellite data. The code has been given to Jeff Whitaker to compare with
EnSRF in the same environment. The differences from these two algorithms
are not significant right now. Emphasis will be on bias correction of
radiance in higher levels. A paper will be submitted in a few weeks.
LETK is planned to be operational in Brazil.

Milija Zupanski:

Work on testing MLEF (maximum likelihood ensemble filter) has begun on
using real observations and satellite radiance data. More attention has
been paid to bias correction by studying the PDF distribution. He will
have more cooperation with people working in satellite data. Dual
resolutions will be tested. A lot of more results from the experiments
will be expected in the next half year. The results will provide a good
comparison with the results from other schemes.

Zoltan Toth:

Scheduled the next tele-conference in December 2006 after Thorpex
meeting in Germany. Grep Hakim will join Thorpex project for regional
ensemble data assimilation. Suggested a new benchmark with higher
resolution summer case. NOAA Thorpex has requested CPU time on Air Force
supercomputers. Proposed a new analysis for shorter lead time of 1-3
hours.

**SUMMARY OF TELECONFERENCE DISCUSSION**

**July 27 2006, 1 pm -- 3 pm EST**

**Present:** T. Hamill (CDC), J. Anderson and J. Tribbia (NCAR), C.
Bishop and Dan (NRL), M. Zupanski (CIRA/CSU), I. Szunyogh (UMD), Y.
Song, Z. Toth and M. Wei (NCEP/EMC) ( E. Kostelich, J. Whitaker and E.
Kalnay on vacation or leave)

**Agenda:**

> 1). Review of progress by each group (with slides if possible)
>
> 2). Plans for the next 3-6 months by each group
>
> 3). Status of benchmark, discussion on unified verification of results

Craig Bishop and Dan:

Showed a new way of doing covariance localization for ensemble based
Kalman filter data assimilation system. They provided 3 slides showing
how to estimate the forecast error covariance matrix using historical
data by modulation. The experiments using this method on a simple system
showed positive results. A full description of this method is expected
at later stage.

Milija Zupanski:

After securing accounts on NCEP IBM computers and experiments have
started to test his MLEF (maximum likelihood ensemble filter). MLEF is
similar to ETKF, except solves for mode (instead of mean) of
distribution. It has tested the simulated and real obs in NCEP
operational environment. Further experiments are planned.

Tom Hamill:

After successfully compared their ensemble data assimilation results
with T62 NCEP SSI benchmark, using same or less amount of conventional
data, some satellite data have been added to the experiments. The
preliminary results are very positive. The pace of further experiments
is delayed by the lack of computer resources. They are looking NCEP for
help.

Istvan Szunyogh:

Started work with real obs, and applied ETKF formulation to local
patches, each grid point at a time. The algorithm is very efficient.
They gave the code to Jeff Whitaker to compare with his square root
filter. Plan to do more research on use of additive inflation
procedures. Studied the effect of imperfect model on DA results, using
bias correction. Two slides distributed showed positive results.

Jeff Anderson (Unfunded collaborator):

No funding from NOAA. Developed a more scalable generic filter, looked
at sampling error in ensemble filters. The system runs an ensemble of
ensembles (4-8) to estimate the error covariance localization factors.
This procedure is very expensive, but only needs to do it once in a
while. They argued that this method of generating covariance
localization factors will make the filter more scalable and more
generic.

Yucheng Song:

Briefly described a new NCEP T62 SSI benchmark analysis/forecast data
set . The new benchmark is based on using satellite data as more groups
are ready to handle larger number of observations.

Zoltan Toth:

Pointed out there is a need for different groups to share some common
verification packages in order to compare different algorithms and
filters. NCEP/EMC will initiate the discussion toward the development of
common verification software..

**SUMMARY OF TELECONFERENCE DISCUSSION**

**July 29 2005, 11 am -- 1 pm EST**

**Present:** J. Whitaker (CDC), J. Anderson (NCAR), C. Bishop (NRL), M.
Zupanski (CIRA/CSU), I. Szunyogh, E. Kostelich (UM), Y. Song, and Z.
Toth (NCEP/EMC) (T. Hamill missed the call, M. Wei on travel)

**Agenda:**

> 1\. Brief description of activities in first year - Each group
> described their work and main results

Jeff Whitaker

Successfully compared their ensemble data assimilation results with T62
NCEP SSI benchmark, using same or less amount of data (in fact, only
75-150k pieces of data were used, compared with \~300k in SSI -- only
data from +/-1 hr window used, and even that was thinned). No radiance,
radar, or scatterometer data used. Results are very encouraging, 5-10%
rms error reduction compared with SSI results (see his slides).
Processing of remotely sensed data with same sequential algorithm is not
practical, looking for alternative solutions (ETKF?) Tested 3 types of
variance inflation methods, difference between successive archived
analysis fields may work best, simple inflation by a coefficient almost
as good

Istvan Szunyogh

Started work with simulated obs, worked well. Adapted ETKF formulation,
still applied locally (region by region), very efficient algorithm. When
switched to assimilation of real observations some bugs got into code,
working on clearing up software. Expects some results by end of summer
2005. Looking into use of additive inflation procedures. Analyzing
effect of imperfect model on DA results, using bias estimation ideas.
Discussed a slide indicating that a relatively small ensemble may be
able to well describe low dimensional dynamics for global circulation
(PECA-type analysis).

Craig Bishop

Was unable to hire post-doc, working with Master level student, had to
adjust research plans somewhat. Worked on producing large ensembles with
ETKF. In parallel, work on generating ensemble perturbations to be
centered around NAVDAS variational analysis, similar to M. Wei's
research at NCEP: use estimate of analysis error variance derived from
NAVDAS to constrain initial ensemble variance using ET algorithm.
Reports that successfully used ET technique to inflate covariance: uses
ET to transform old archived ensemble data tfor inflating variance in
tropics of current ensemble Toth points out link between this work and
that of D. Hou at EMC who plans to use similar technique to introduce
stochastic perturbations. Plans to experiment with combining ensembles
from different sources.

Milija Zupanski

Hired postdoc, secured accounts on NCEP IBM computers. Plans to test his
method, similar to ETKF, except solves for mode (instead of mean) of
distribution. Currently setting up software on NCEP machines, will start
testing with simulated obs soon, in couple of mos will start using real
obs. Plans using bos operators and other applicable software from NCEP
SSI code. This will enable quick technology transfer to NCEP operations
if research is successful.

Jeff Anderson (Unfunded collaborator)

Made an attempt to port GFS system to NCAR. Work is not complete, no
funding from NOAA. Worked on generic filters, looked at sampling error
in ensemble filters. Found a solution where no inflation is needed in
perfect model setup. Ran some experiments, without much tuning, with
NCAR T85 CAM climate model, real observations, January 2003 cases, using
radiosonde and other traditional data, but no radiances. Compared
results with GFS T254 operational system (p. 16 of his slides). Very
encouraging results, 5-10+% rms error reduction for temperature, even
larger reduction for low level wind errors. Problem with winds higher up
traced to use of inaccurate obs error variances with ACAR data.

Yucheng Song

Briefly described NCEP T62 SSI benchmark analysis/forecast data set that
he prepared for use by other groups (see below)

Zoltan Toth

Pointed out few links between external research and NCEP development
activities: Connection between model error studies of B. Hunt (UM) and
M. Pena (EMC); ET initialization by C. Bishop (NRL) and M. Wei (EMC);
Inflation with ET method by C. Bishop (NRL) and D. Hou (EMC).

> 2). Preliminary discussion on plans for 2^nd^ year

ZT commented that the results by JW-TH & FA are very encouraging, and
warrant continuation of ensemble-based DA research work. There was
general agreement on this. JW and CB discussed potential for using
ensemble covariance information for improving variational schemes. They
pointed out the demonstrated ability of variational schemes to process
large amounts of data. JA made the point that ensemble-based DA is a new
field and there is no evidence that these methods could not be modified
to cope with heavy data volume, all agreed on this. MZ mentioned that
after working on 4DVAR for 10 yrs, he switched to ens-DA methods because
he believes they offer a theoretically more appealing approach. IS & ZT
pointed out that CPU limitations on current operational machines should
not constrain research aimed at 3-5 years into the future. Focus should
be on understanding whether and how much improvements can be gained by
using ens-DA methods compared to variational methods. Algorithms should
be built with resource limitations in mind, but that should not be the
primary consideration at this stage. Optimization of procedures can be
considered and will become more important as the research evolves. ZT
suggested each group to continue their work under their proposal, and
the project to keep focusing on ensemble-based data assimilation
methods. Work on hybrid methods (where information from a set of
ensemble members are used in variational DA) is encouraged but the
THORPEX ens-DA funds should support the development and testing of
ensemble-based DA schemes. This research has a horizon of 3-5 yrs, as
compared to hybrid applications that if funded through other mechanisms
may bring some benefits on shorter time scale.

> 3\) Collaborative work within the project

ZT discussed the possibility of JW-TH, beyond their own research,
playing a central role in trying to build prototype ens-DA system as
time goes that would include useful and new results from any of the
participating groups. This will be further discussed at next meeting.

Proposed dates/time for next meeting:

Sept 7, 1-3 pm eastern time

Sept 9, 11 am -- 1 pm eastern time

Proposed agenda:

1)  Review detailed plans for yr2 (each group present their plans)

2)  What should be our stated goal for yr2 as a group? Like for first
    year, we wanted to generate a benchmark, have initial comparison;
    what should we aim to accomplish by end of yr2?

3)  How to enhance collaboration?

**The NCEP T62 Benchmark run**

Yucheng Song

This document summarizes the benchmark experiment done at NCEP in
preparation for the inter-comparison of different ensemble-based data
assimilation schemes.

**MODEL VERSION**

To be comparable to the four independent groups that work on
ensemble-based data assimilation (EBDA), we used the executable of
global forecast model (named global_fcst6228, which is Triangular
truncation T62 with 28 levels) archived at NCEP high performance storage
system (HPSS). For the assimilation, we used the executable compiled on
March 2 (which is named as global_ssi). For interested users who have
accounts at NCEP, the source is

/hpssprod/runhistory/rh2004/200405/20040530/nwprod.tar

**TEST PERIOD**

From January 1 to the February 29 of 2004, the first \~15 days are also
archived, thought they might be excluded from the evaluation.

**EXPERIMENT PLATFORM**

The benchmark experiment is finished on NCEP IBM BLUE machines.

**OBSERVATIONS TO BE USED**

The post quality control (post-qc) files are used for the experiment. A
note here is that the data has been processed by comparing with the
high-resolution gdas guess files. The input data files used are archived
onto HPSS as well which is named
/hpssuser/g01/wx20ys/Benchmark/dump.tar. The file also contains SST, ICE
and SNOW data files used for the experiment.

**ARCHIVED FILES**

Every 6 hours, for the 00Z and 12Z cycle, pgb (pressure level grib)
files are archived on the NCEP HPSS system. Bias correction, satellite
angle, surface analysis as well as sigma analysis files are also
archived. There are 31 levels in the pgb file, they are:

1000 975 950 925 900 850 800 750 700 650 600 550 500 450 400 350 300 250
200 150 100 70 50 30 20 10 7 5 3 2 1mb

**HOW TO GET THE FILES**

Data are archived on HPSS by day, for example, to get the data file for
Feb 24, in your desired directory, you can issue command like --

hpsstar get /hpssuser/g01/wx20ys/Benchmark/20040224.tar

The command hpsstar is Mark Iredell's version of tar which is convenient
to use. Users can also use htar to get the files.

**QUESTIONS or COMMENTS**

If after going through this entire document, you still have questions,
please let me know. I can be reached at yucheng.song@noaa.gov.

**SUMMARY OF TELECONFERENCE DISCUSSION**

**April 7, 2004, 2-3 pm EST**

**Present:** J. Whitaker, T. Hamill (CDC), J. Anderson, J. Tribbia
(NCAR), C. Bishop (NRL, joined later), M. Zupanski (CIRA/CSU), I.
Szunyogh, E. Kostelich (UM), M. Wei, R. Wobus, Y. Zhu, and Z. Toth
(NCEP/EMC)

**General issues:**

1)  Funding

ZT mentioned that two participating groups will likely receive more
funding than initially thought, restoring funding level to that
originally requested by one group.

2)  Use of real observations or simulated observations in a perfect
    model framework?

ZT emphasized the main focus should be on inter-comparison of different
methods using real data. Several participants pointed out science
advantages of carrying out perfect model data assimilation experiments
as well. It was agreed that if resources permit, the project would
include perfect model DA experiments as well. Rest of the discussion
focused on real obs experiments, since this is the primary interest from
NCEP's point of view, and this is the setup that should drive basic
experimental design etc. It was noted that the addition of perfect model
DA experiments would not double resources needed for the project. JA
noted that NCAR could generate data based on a model integration. ZT
pointed to the NCEP OSSE software that should preferably be exercised
for generating simulated data (with realistic observational error).
Participants (including EMC) are asked to assess whether the addition of
perfect model experiments is within their reach. In case it is, details
of the perfect model setup will be discussed after plans are fleshed
out, and work begins with real observations.

3)  Software infrastructure to be used.

ZT recalled that whenever possible, software available from NCEP (NWP
model, observation operators, file format, verification routines, etc)
should be used. In case new software needs to be developed in the
inter-comparison project, it should be compatible with existing NCEP
software and practices. These practices will insure that as the project
progresses, participants can easily exchange parts of their software,
can start working jointly on a prototype software, that can later be
tested in an operational environment.

**Experimental design:**

1)  Test period

After short discussion, participants agreed to use Jan-Feb 2004 as a
test period. The first \~15 days will be excluded from the evaluation.

2)  Observations to be used

*Data types.* After some discussion, participants agreed that in the
main experiment, the following data types will be used:

Surface observations, radiosondes, ACAR winds, cloud drift winds.
Participants can ignore some of the observation types as they wish.

*Data files.* The NCEP prebufr files from the final gdas analysis cycle
will be used. EMC is going to make the gdas1 prepbufr files (including
restricted access data, please confirm you have privileges to use that,
and whether you can all use blocked data format) available on the IBM
machine in a few days. There was some discussion about using CDAS data
files. These files use a much longer data cut-off time that does not
allow for their use in real time weather forecasting. Also, they may not
contain some new data types that in further analysis we may want to
consider.

In addition to the basic experiment, participating groups can also run a
second experiment where they include additional data types.

*Observational period.* Following the usual practice (also reflected in
+/-3 hrs time window for data included in prepbufr files) analyses
performed at the nominal 0000 UTC time, for example, will use data up to
3 hrs after the nominal analysis time (0300 UTC). It was noted that
unlike 3DVAR where through time interpolation, "future" data are used
(currently up to 3 hrs into the future), ensemble-based schemes are
filters that may use data only up to the time of the analysis.
Therefore, if participants desire, they can choose to perform an
analysis step at 0300 UTC (using data up to that valid time), for a
comparison with SSI forecasts initialized at 0000 UTC (that also use
data up to 0300 UTC).

Issues not discussed/settled yet:

*Observational error statistics.* This has not been discussed*.
Suggestion* -- use observational error statistics as used in operational
3DVAR, given in prepbufr files.

*Quality control.* This issue has not been discussed yet. *Suggestion*
-- use operational QC marks as given in prepbufr files. Agree about
cut-off value regarding QC mark below/above which data will/will not be
used.

3)  Data assimilation

*Cycling frequency:* Each group decide on their own. *Suggestion* -
required minimum analysis frequency every 6 hrs (available at 00, 06,
12, and 1800 UTC).

**Next meeting, tentative:** Monday, 11 April, 10 am Pacific, 11 am
Mountain, 1 pm Eastern time

Continue with discussion of remaining experimental design issues in
strawperson plan.

**INTERCOMPARISON OF DIFFERENT ENSEMBLE-BASED DATA ASSIMILATION SCHEMES
(Jan. 21, 2004)**

**GENERAL DESCRIPTION**

**BACKGROUND.** Currently there are four groups that work independently
with different ensemble-based data assimilation (EBDA), or ensemble
generation schemes using the NCEP global assimilation/forecasting
software:

*Jeff Anderson and David Parrish* -- developed EBDA software (Ensemble
Adjustment Filter, EAF) and are in the process of testing it with T62
GFS system

*Mozheng Wei et al*. -- developed/adapted ensemble perturbation software
based on EBDA concept using Ensemble Transform Kalman Filter (ETKF)
algorithm, in the process of testing initial perturbation scheme. This
scheme will first be tested only as an initial perturbation generation
tool. Later it can be adapted and tested for data assimilation
applications, too.

*Istvan Szunyogh et al.* -- developed EBDA software (regional variant of
ETKF, RETKF) using GFS model, tested in perfect model environment, in
the process of setting up software to assimilate actual observations

*Whitaker and Hamill* -- developed EBDA software (Ensemble Square Root
Filter, ESRF, related to EAF), and tested it using limited surface
observations

**OBJECTIVE.** Compare the performance of the different schemes in terms
of the quality of their data assimilation and ensemble initial
perturbations results. Identify the strengths and weaknesses of each
scheme.

**WORK SCHEDULE**. The inter-comparison part of the project will be
completed in 12 months, depending on available resources. Suggested
starting date is Febr. 1, 2004.

Febr. 04 Work plan finalized

March-May Prepare software

June-Oct. Run (and if necessary, rerun) experiments

Nov.-Dec. Verification/evaluation

Jan. 05 Short summary report, with recommendations for future work

**LONG TERM BENEFITS**. The most promising elements identified in the
different EBDA schemes will be used to construct an EBDA scheme to be
further developed and later tested in a quasi-operational environment.
It is anticipated that most or preferably all groups will contribute to
the design of a scheme for future development.

**PROJECT CREDO**. The four groups will seek agreement on the specifics
of an experimental design that each group can accept and adopt, allowing
for a fair comparison of the different techniques. The four groups will
agree to use a common set of values/procedures for several important
aspects of the experiments. The use of a *common experimental design*
will allow the identification and attribution of differences in
performance that arise due to the differences between the various *EBDA
schemes themselves*.

**WORK PLAN.** Each group will make the necessary modifications to their
software and will perform the experiments on their own computers.
Verification software and benchmark experiments will be provided by
NCEP.

**EXPERIMENTAL DESIGN**

**BACKGROUND.** The specifics below are based on initial discussions
with some project participants and are presented here only as a starting
point for discussions. The four groups are expected to develop and agree
on the details of the inter-comparison study.

**EXPERIMENTS** *Test period:* 2-month period from most recent winter
(Jan-Febr 2004?)

**OBSERVATIONS** *Data set:* NCEP operational prepbufr and satellite
data files, containing data in +/-3hr chunks

> *Data selection*: There are two options here. Each group uses (1) all
> data they can; (2) the largest set of observations that every group
> can assimilate (ie, model variables with height assignment)?
>
> *Error statistics* As given in operational data files

**DATA ASSIMILATION** *Output frequency:* 6 hourly?

> *Model error*: Common approach (eg, multiplicative noise), or allow
> different treatment by each group?

**FORECASTS** *Model version*: NCEP GFS, operational at start of project

> *Resolution:* T62L28

*Digital filter*: Turned off?

**ENSEMBLE** *Size*: 50-100 members (40?)

*Surface fields*: Same for all members?

*Frequency/length:* Once per day (00Z), output every 12 hrs out to 7.5
(or 16?) days

*Output format:* Enspost files

*Output variables:* 1000 & 500 hPa geop. height, 850 hPa temp, u&v winds
at 850 & 250 hPa, and precip

**VERIFICATION** *Data assimilation* Fit of mean of first guess
forecasts (in sigma or standard pressure GRIB file) to observations,
based on standard statistics used at NCEP

> *Initial perturbations* Reposition initial ensemble members to be
> centered around operational SSI analysis. Run set of ensemble
> forecasts and evaluate them using standard NCEP probabilistic
> verification measures

**BENCHMARK** *Data assimilation*: Operational SSI analysis, run at
equivalent resolution

> *Initial perturbations*: Operational bred perturbations (at equivalent
> resolution