**Meeting on the New Nature run and future OSSEs**

1pm-3pm June 9, 2006, Rm 209, WWB

Attendee

Lars Peter Riishojgaard (LPR), John LeMarshall (JM) , Oreste Reale (OR),
Dave Emmitt (GDE), Sidney Wood, Joe Terry, Ronald Errico, Juan Jusem,
Zoltan Toth (ZT), Wayman Baker, Jack Woollen (JW) , Yuchang Son, Haibing
Sun, Michiko Masutani (MM)

VTC

Thomas W Schlatter (TWS), Steve Weygandt Prashant D Sardeshmukh

Tel

Gilbert P Compo

**1. Progress in NR integration at ECMWF**

**Geopotential height in Model Levels will be added**

> ID=100: tentative
>
> Codes has been developed and tested

**Pressure Level Data**

> 31 Levels
>
> Additional 10 levels to ECMWF current routine archive
>
>  975 950 850 700 650 600 550 450 400 350
>
> Scripts has been tested
>
> ECMWF Archive: full resolution
>
> 1x1 : Committed to be provided to US
>
> Accumulated fields will be archived over short periods, then reset.
>
> Code to add this feature as an option has been developed and tested.

**Isentropic Level Data**

> Levels: 315K 330K 350K  370K 530K
>
> 315K midlatitude influence to tropical cyclone generation.
>
> Interest from AMMA (African Monsoon Multidisciplinary Analysis)
>
> 330K Blocking
>
> 350K 370K near tropopose and front
>
> Can produce IPV at 360K if PV is EPV
>
> 530K stratosphere
>
> Variables\
> PV, VO, DiV,  P, Q, O3, Mont\
> \
> *Complete data will be processed by Don Johnson if resource become
> available*

**A pre-cursor run in preparation for the NR.**

> **\
> **\
> T159  L91,\
> From 1-May Whole year
>
> IFS cycle 30r1, (31r1 will be used for NR)
>
> exp=erz3.
>
> Half of our new machine sometime within the next few weeks.
>
> The Nature run will be produced with 31r1 version of the forecasting
> system. There is now a stable configuration of this code-release. The
> pre-operational testing is about to restart. The seasonal forecasting
> group has evaluated its performance, and are happy with it (good
> news!) The modeling group are also happy with it, and it will be the
> basis for the so-called interim re-analysis (to start in the autumn).

**2. Media for transporting the Nature run**

Current plan is to use hard disk.

Make sure the hard disks ECMWF use will be accepted by NCEP, NASA, and
ESRL.

We will try to find out the type of the disk as soon as possible..

Check the security issues.

If grib1 is used with simple packing the volume will increase.

It may be easier to pay for more disk than struggle with packing

ESRL will try data transfer through HSI first before using hard disks.

Gil Compo offered to help with software needs.

**3. Grib code**

We agreed that Grib2 will be the basic format for OSSEs but grib2
require development in both NCEP and ECMWF.

NASA is working on NCAR lib which uses ECMWF grib1 decoder. (JT)

NCEP decoder

Complex packing spectral grib is supported for grib2

Grib1 and grib2 Convertion is not supported for spectral component

Complex packing in grib 1 require implementation

Spectral data in grib1 has limited support

Wgrib and grads for grib2 is under development and will take some time
to be completed.

Progress

Sample grib2 with spectral data with complex packing has been provided
by ECMWF to NCO for testing.

ECMWF data and software support section and NCO are working together.

**4. Software for spectral harmonics to grid conversion**

Steve Weygandt said Splib from NCEP works well

NASA is trying to use NCAR spherical harmonics code. (JT)

**5. Listing of tasks before start OSSEs**

**5.1Evaluation of Nature run**

Frequency of cyclone

Cyclone tracks.

Looking at sclae of motion

JM said, aquirering ECMWF evaluation to their own model is the first
thing to do.

JW emphasized that evaluation of the nature run is very important and
rest of the work depend on careful evaluation.

Juan proposed event oriented diagnostics. He will provide a note with
Oreste and Joe later.

**Comments**

Event oriented evaluation proposed by OR, Juan and JT is attached.

**Progress**

ECMWF will provide monthly routine diagnostics for the nature run. This
is a set of monthly diagnostics viewed internally by ECMWF and to be
posted public web site in near future. Sample diagnostics for T159 run
for June 2005 is posted at OSSE web site.

jpg files are posted at\
<http://www.emc.ncep.noaa.gov/research/osse/NR/ECMWF.diag.159.sample>\
Ps files are tared and compressed as\
<ftp://ftp.emc.ncep.noaa.gov/exper/mmasutani/ECMWF_NR_Sample/ECMWF.diag.159.sample/NR_vs_Clim_200506.tar.gz>

**5.2Selection of focus period**

ZT suggest to make a list of criteria

LPR said Jet is most important

Hurricane

High impact weather

There are argument between one 6 week or two 3 week.

> 6 week is necessary for any reasonable OSSEs. With two 3 week we may
> get two useless OSSEs
>
> We need winter storm and hurricanes. It is hard to choose one.
>
> If we ask entire cpmmunity to selct period we will not be able to
> agree any thing. Six weeks may cover few events which satisfy more
> people.
>
> Tom Schlatter said we need two 6 week high resolution nature runs.

Oreste said, evaluation of the NR comes first. We can discuss about the
selection later.

**5.3Development of forward models**

> GMAO will work on if the fund become available.
>
> JM said we should make use of CRTM. Both simulation and assimilation
> should be using CRTM.
>
> JM said CRTM have option between OPTRAN and RTOVS.
>
> Haibing is using UMBC model to simulate AIRS and he has already
> simulated CrIs, We need to think about how these experience will be
> integrated to development of CRTM.
>
> Comments
>
> THORPEX funding has been provided to SWA and AOML to develop an OSSE
> framework for THORPEX. This effort includes funds for identifying
> existing forward models for all current observing systems. These
> forward models will need to be adapted to Nature Run inputs including
> modified cloud optical properties which SWA would be responsible for
> (under the THORPEX funds).
>
> JM mentioned UMBC model should be included in CRTM
>
> Comments from Ron Errico is attached.

**5.4Any work to be done for the data.**

> Conversion for NCEP grib if necessary
>
> Gil Compo agreed to help any work needed.

**5.5 Simulation of basic data (make the list for each category)**

GDE commented that Air craft data and CMV depends on weather.

No one has yet simulated aircraft data, taking into account that flight
routes tend to steer clear of thunderstorms or that flights are
cancelled entirely when large storms affect busy airports.

MM suggested that we should not use these data for calibration.

LPR said, AMSU is good to be included in calibration

NOAA 18 is good to be included.

Michiko suggested idealized data is useful to compare DA systems

**7. Listing possible future observational data to be simulated**

LPR commented there are two kinds of future observisng systems.

The systems under planing and development

The systems considered as potential

JCSDA is listed

CrIS, ATMS, GOES-R/GIFTS , HyMS\* -- P and G%.

\* HyMS Hyperspectral Microwave Sounder

\% P- Polar, G Geostationary

ESRL has interest in

> UAS: Unmanned Air System

Other interest

> DWL: Space
>
> UAS
>
> Targeted Observation
>
> Targeted assimilation

**Comments**

JM mentioned high resolution (1km) soundeing ARIES (Please expand) from
JPL. 1km resolution data will reduced to one observation per every 55km
square if cloud clear area is selected.

More propsed instruments are listed in attachment B from SWA.

GMAO will add more list

**8. List of metric of evaluation of data impact**

Anomaly correlation

Different spectral domain

Hurricane track

Spectral decomposition of obs-ges by RE was good

**9. Discussion about whether OSSEs will use GSI or SSI**

NCEP is trying to keep OSSE and operational system parallel

GSI is being updated regularly

Need frequent selected calibration

SSI is somewhat frozen in Feb2005

CRTM is being updated

GSI has additional features

Take height as vertical coordinate

More diagnostics output

LPR and JM strongly insist GSI. ESRL concurs.

Comments

John Derber mention GSI is up date every three month.

**10. Forming Sub groups**

WB and ZT suggested for form subgroups be formed to establish and
distribute tasks.

> a\. Evaluation of the Nature run
>
> (including selection of window(s) for high resolution enhancements)\
> b. Adjustments to the Nature run\
> c. Simulation of observations (existing and new types)\
> d. Calibration and evaluation metrics\
> e. Design of OSSE experiments

LPR suggested group "b" should be combined with group "a."

Comments

Need volunteer for coordinating activities for each of these themes?
Several people will contribute to each of these hopefully. We should not
make it too formal. (ZT)

a.b.c.d is for preparation "e" is actual OSSEs.

For group "c", simulating observations for calibration is part of the
preparation, but simulation of future observations is part of actual
OSSE. We cannot mix these two tasks. (MM)

**11. Thorpex**

GDE tpointed out that there is a THORPEX fund dedicated to OSSE and we
need to make use of it.

**Note.**

Yuanfu Xie of ESRL will spend about six month at NCEP to help
preparation of OSSEs. Later, he will work on OSSEs for UAS.

**Attachment**

**A. NR evelation strategies proposed by NASA OSSE team**

\- Evaluation of the realism of the NR by assessing the likelihood of\
  extremes that are beyond analyzed or measured values (using Gumbel\
  plots).\
\
- Analysis and forecast of precipitation (threat scores, etc).

\[Comment from Tom Schlatter: Not sure how one can

judge accuracy of precipitation ("threat scores") in the nature run
because the nature run is defined as "truth." May be able to judge the
*realism* of nature run precipitation by comparing with real-world
precipitation patterns.\]

\[ Response from Oreste Reale: Tom Schlatter is right and we thank him
for pointing out the lack of clarity of our statement. We did not
clarify that all the threat scores are to be performed on cycles of data
assimilation and forecasts, using the nature run as \'truth\'. We then
want to compare them with threat scores coming from forecasts performed
by the same forecast model but on real cases, validating the forecast
against obs precip. With a statistically significant sample of cases
taken on various precip events, this could be an indirect way of
assessing the realism of the nature run precip with respect to obs
precip.\
Hope this addresses his right concern. \]

\- Perform a scale-of-motion analysis in which we will extract
wavelength amplitude and propagation speed of disturbances over
vertically-integrated layers. We will examine geopotential and wind
fields at 3 layers: low-level (sfc to 700 hPa), mid-troposphere (700-400
hPa) and near-tropopause (400-100 hPa).\
\
- Build a 3-d diagram of amplitude, wavelength and propagation speed (or
phase) in which the fundamental types of disturbance, be they tropical,
mid-latitude, hybrid or transitioning, should be identifiable. This
approach can be used for slices of reanalyses and/or operational
analyses of length comparable to the Nature Run (not necessarily the
same year), chosen perhaps among periods of near-neutral and/or strong
ENSO positive/negative phases, to provide a larger range of
variability.\
\
- Comparisons with may shed light on the realism of disturbances in the
model and pinpoint possible unrealistic or spurious scales of motion.

**B. The following are SWA's possible contributions to the five
sub-groups identified above.**

a\. Evaluation of the Nature run

(including selection of window(s) for high resolution enhancements)

SWA could assess NR clouds (as it has done for all previous NRs). This
would include review of ECMWF publications of model cloud evaluations.

SWA could assess the NR parameterization of sub-grid scale wind
variability. This is needed for scatterometer and DWL simulations

SWA could generate the tables needed to distribute aerosols in a
quasi-local manner within a climate envelop.

b\. Adjustments to the Nature run

Based upon the findings in the NR evaluation, SWA would provide adjusted
cloud fields, background aerosol fields and any sub-grid scale wind
variability adjustments

QUESTION: In the past, SWA has generated a GADS version of the NR that
was suitable for simulating lidar and CMV observations. TOVS
observations were not done with the same cloud fields (I believe).
Should we produce an adjusted NR to be used by all forward models, AIRS
especially?

Comments

> Please expand for unfamilier acronym (TWS)
>
> The GADS is an SWA created direct access
>
> database that contains the nature run plus many fields that are not in
>
> the original nature run. Some of the fields are unique from a
>
> space-based satellite view point.
>
> http://www.swa.com/ald/lsm/index.html
>
> (Sid Wood)

c\. Simulation of observations (existing and new types)

SWA could provide simulations of ACARS, CMV, WVMV winds for "current
observations".

SWA could provide simulations of future observing systems such as DWL,
DIAL and sub orbital platform sensors (UAVs, rocket sondes, balloons,
etc)

d\. Calibration and evaluation metrics

SWA could work with the OSSE group to assemble a list of metrics for
evaluating the NR and the output of the OSSEs. The DWL WG has a list of
nine such metrics for assessing the impacts of DWL.

e\. Design of OSSE experiments

THORPEX has provided funding to help establish a "science" oversight
panel that would help to set priorities for OSSEs and expand the
participation to academia, other US agencies and international groups.
Zoltan is familiar with this THORPEX task.

It makes sense for the JCSDA to host this OSSE panel.

**C. Comments from Ron Errico**

I think it is important that all members in the OSSE group have a common
understanding about certain critical aspects of OSSE experiments.  For
this reason, I think it helpful to open up some of these issues for
either explicit recognition or debate.\
\
1. For all observation types currently being assimilated forward models
of some sort are already available, because they are employed within the
data assimilation system.  They may not be the best forward models and
perhaps some aspects are even crude, but they exist and have a degree of
proven utility.  The simulated obs, however, may be most realistic with
respect to the nature run if they are determined using the best forward
models we can design.  Subsequently, we can\
also consider whether we want the OSSE DAS to use some of these better
forward models.  But, using different forward models creates some other
problems for consideration, as described below.\
\
2. More difficult, is to create realistic observation and forward model
errors, especially biases or gross errors. The problem is, we really do
not know what these error characteristics are for many\
observation types as well as we would like to know them.  Also,
typically what we call observation error statistics are actually
composites of forward model (including representativeness) and
instrument error.  As we change forward models compared to those
employed in the data assimilation, the contribution by the forward model
error will be changed.\
So, as we develop forward models to simulate the observations, we must
consider the implications for how we specify the subsequent observation
errors.\
\
3. Specifying these characteristics is therefore as much art as science
at this time. The more sophisticated error characteristics we try to
incorporate, the more validation will be required. In particular, in
past attempts, it has been very difficult to add gross errors and error
biases to observations in ways that yield quality control decisions in
the OSSEs that have equivalent characteristics to those in corresponding
real data assimilation systems.  For example, the\
OSSE should have similar observation acceptance rates and geographical
distributions as in real assimilations with each data type.  There are
work-arounds for achieving this result, such as adding no gross errors
or biases to the simulated observations and relying on quality control\
decisions provided by a corresponding (in this case one-to-one) real
data. With this technique, however, it may be inappropriate to use very
different forward models than already in the assimilation, since any
really significant difference in the models likely produces biases\
and some gross errors which will remain undetected as the quality
control is rendered independent of the OSSE innovations (and only
dependent on the innovations in the real system)\
\
The point is the issues of forward model selection, observation error
design, data selection criteria, and quality control in the OSSE are all
connected.  Although OSSES have been performed in the community for a
long time, the importance of these issues have often been\
neglected, with the result that many researchers find the results
untrustworthy. So, we should provide the attention they deserve.  It
requires considerable understanding of the observations as well as the
DAS that determines how the data is used.

D. ESRL contribution preview\
ESRL Global Systems Division will commit to sending Yuanfu Xie to EMC on
a six-month visit starting around 1 Oct 2006.  ESRL/GSD will pay his
salary and travel expenses while he is on assignment at EMC.  His will
become famiiar with EMC OSSE procedures, the EMC computing\
environment, and the GFS and GSI codes that will be used for OSSEs.
After he has \"spun up,\" He may be able to assist on one of your OSSE
related projects.  By the time he returns, we hope he will be equipped
to assist us in our own OSSEs, following EMC practices.\
\
ESRL/GSD is interested in performing an OSSE whereby unmanned aircraft
systems (UAS) would sound the atmosphere at a regular set of points and
at regular intervals of time, to augment existing climate and weather
observing systems with high-resolution (in the vertical) in situ
observations over the oceans and polar regions.  We would simulate the
UAS observations necessary to perform this OSSE.

E. Roles of NCEP in OSSEs (suggestion by MM)

Since NASA, ESRL and SWA are writing about their contribution to OSSEs,
I thought NCEP nave to say some thing as well. I jot down few lines. Any
comments appreciated.

NCEP contribution to OSSEs

NASA, SWA, ESRL, and JCSDA will have specific instruments to promote. A
main role of NCEP will be verifying the claims made by each group.

Save NR in the format NCEP grib decoder can handle. Test decoders and
spherical harmonics software.

Maintain parallel structures between OSSEs and operational analysis.
Keep testing the update in GSI and CRTM.

Comparison between satellites based observation and non satellite
observation.

Evaluation of ensemble based targeting. (This could be done by other
institute such as ESRL)

**F. Grib decoder (Joe Terry)**

I just want to update you on my attempts at processing the EC nature run
sample data.\
So far, I am able to convert the T799 model level temperature field from
spectral\
to grid point using the NCAR conversion software.   The code was
terribly\
hardwired and slow but I think it will be usable.  It should run much
faster with the\
T511 data.  Since the EC model has hybrid levels, I was able to compare
the model\
level data with the pressure level data in the stratosphere.  The model
level data\
matches very well, so I think it\'s working correctly.\
\
Any word yet on the type of hard drive EC is sending?