Graphics Products from the NCEP Medium Range Ensemble
(16 November 1995)

Note that the graphics files described here are in a state of
continuous development, and that when we make changes to the
graphic files in N-AWIPS we will update this document
accordingly.

Please remember that these products cannot be used in any way
for commercial purposes.

SUMMARY OF CURRENT PRODUCTS:

The following products are produced twice a day in files named
ens_yymmdd_00 and ens_yymmdd_12:

Spaghetti diagrams (12, 36, 60, ... 252 hr for 00z runs):

500mb height:            5220m (cool season only, Oct. thru Apr.)
                         5400m
                         5520m
                         5640m
                         5760m
                         5820m
                         5880m (warm season only, May thru Sep.)
500-1000mb thickness:    5400m
850 temperature:          0 C  (cool season, Oct. thru Apr.)
                         15 C  (warm season, May thru Sep.)
MSL pressure:            992 mb (cool season, Sep. thru Apr.)
                         1000 mb
                         1008 mb (warm season, May thru Aug.)
                         High and Low centers
                         1024 mb (warm season, Mar. thru Nov.)
                         1032 mb 
                         1040 mb (cool season, Dec. thru Feb.)
250 mb winds:            50 m/s (varies by month from 25 to 50)
850 mb winds:            15 m/s (varies by month from 12 to 15)
700 mb RH:               90%

Note that the seasonal contours are chosen based on the month in
which the forecast starts, and that warm and cool season months
are defined separately for each variable.

The following products are prepared at 00Z only, in files named
ens_yymmdd_00_cluster:

Cluster products (84, 108, 132 hr):

Ensemble mean:      500mb ht, 1000mb ht, 500-1000 thickness
Ensemble spread:    500mb ht, 1000mb ht
MRF:                500mb ht, 1000mb ht, 500-1000 thickness
Each Cluster:       500mb ht, 1000mb ht

Each map except spread includes the anomaly in dashed lines.
Probability products (72, 96, 120, 144 hr):

500mb height:                 5580m, 5640m, 5700m
500mb height 24hr tendency:   +30m, -30m
500-1000 thickness:           5400m, +60m anomaly, -60m anomaly
850 mb temperature:           0 C
700 mb RH:                    70%

AVAILABILITY

The jobs that prepare data for these graphics are not yet fully
operational, so the products may be delayed or unavailable from
time to time.  The most recent two days of GEMPAK graphics
metafiles are generally available.  They may be viewed with
ntrans (available from Computing Development Branch of NCEP).

LOCATIONS OF METAFILES:

N-AWIPS servers:

In ntrans, bring up the model selection panel and select the
model ENS.

nic server:

The metafiles can be downloaded by anonymous ftp from
nic.fb4.noaa.gov in the directory /pub/nadata/meta/model/ens by
the following procedure:

ftp to:
nic.fb4.noaa.gov
For userid, type:
anonymous 
for password, use your full ip-address.  Then cd
pub/nadata/meta/model/ens.  There you will find 3 files for each
day.  Two files contain "spaghetti" plots as described below. The
other file (_cluster) contains additional plots for the ensemble
mean, cluster means and probabilities derived from the ensemble. 

Experimental versions at NCEP:

Experimental versions of these products are available to users
within NCEP in the directory /export/sgi80/wd20rw/meta .  The
easiest way to view these files is to cd to this directory, then
enter ntrans (note that if you want to print these graphics, you
should instead be in the directory where you want PostScript
print files written).  Then choose the entry for user-defined
files in the ntrans open-file menu.  If you did not change to the
metafile directory first, you will need to enter the directory
name followed by /* in the upper entry area, then choose search. 
Categories of metafiles can be identified by one of the following
suffixes:
     _t  test:  current version of code to be implemented in N-
AWIPS, serves as a backup to N-AWIPS.
     _u  alternate:  contains out-of-season contours
corresponding to the test version.
     _x  experimental:  contains whatever new products we are
testing at the moment.
     _v  verification products, currently rerun 3, 4, 5, and 10
days after the forecast start time.

In the very near future the experimental products will be
completely reorganized.  After they have been tested and
commented upon, some of the new set of products will become
operational.  Each file will be identified by a series of 2 or
more suffixes:
     region: _us _na _sa _nh _sh _car _atl
     product: _pr _en _cl _sp _xx (probability, ensemble mean and
spread, clusters, spaghetti, experimental)
     data source:   _e ecmwf ensemble
                    _i international, using multiple centers
                    (none) ncep ensemble
     verification:  _v verification
                    _vn verification through n days
                    (none) no verification
     contours:      _a alternate, all seasonal contours
                    (none) standard, varying by month
     version:       _x experimental
                    _t test
                    (none) operational
 
  example: ens_951103_00_us_pr_v3_t test version of probability
on US map with verification through 3 days.  Note that in the new
scheme experimental versions of current products (_x) are in
different metafiles from experimental products (_xx).  

DESCRIPTIONS:

Spaghetti diagrams:

Each diagram shows the same one contour of one meteorological
field from each of the 17 runs making up the ensemble.  In
addition to the T126 MRF and its T62 extension (shown in yellow)
and the 12Z T126 AVN run with its T62 extension (shown in green),
there are 10 perturbation runs (shown in red) based on adding and
subtracting 5 perturbation fields from the CNT run (T62 MRF,
shown in orange), and 4 perturbation runs (shown in blue) based
on 2 perturbation fields applied to the 12Z AVN.  

The "Control" runs are plotted on top of the corresponding
perturbations, and all runs from the most recent cycle are
plotted on top of runs from 12 hours earlier.  The principal
control run in the current cycle (MRF or AVN) is plotted with a
wider line wherever it does not cross over other lines.  Counting
the MRF, the T62 MRF (CNT), and AVN, there are 17 runs in all. 

Probability:

The probability fields are generated by counting at each point
the number of runs in which, for example, the 500mb-1000mb
thickness exceeds 5400m, rescaled to a percentage.  For fields
like 700mb RH, for which the spaghetti diagram becomes almost
unreadable after a day or two, this is basically a more legible
way to present some of the same information.  

Currently, for historical reasons, the probability products are
produced for 00Z verifying times while all the other products are
produced for 12Z.  This will be changed in the near future.

The generation of probabilities is unweighted;  that is, all the
runs have equal weight.

Ensemble means and spreads:

The ensemble means are weighted, using weights recalculated every
few weeks.  The weights are empirically determined for 5 groups 
of ensemble members.

The anomaly fields shown in dashes on the ensemble mean maps are
currently based on Climate Prediction Center databases covering
the years 1979 thru 1988 (500mb) and 1979 thru 1985 (1000mb).  We
expect to switch to a longer period database. 

The spreads (standard deviation of the ensemble members about the
mean) are shown without normalizing for the climatological
variance.  This means the spreads tend to increase with latitude
along with the amplitude of climatological variance of height. 
We expect to add a product with normalized spreads in the future. 

MRF maps:

The MRF run is shown on maps analogous to those on which the
ensemble mean and the clusters are shown.

The anomaly fields shown in dashes on the MRF maps are currently
based on CPC databases covering the years 1979 thru 1988 (500mb)
and 1979 thru 1985 (1000mb).  

Clusters:

The 17 runs are grouped into up to 6 clusters.  Currently the
clustering is done separately for each of 3 forecast hours and 2
levels, but in the near future it will be changed so that all
products will be based on clustering of 500mb heights at 132
hours, which will make it possible to track the evolution of each
cluster.

The anomaly fields shown in dashes on the cluster maps are
currently based on CPC databases covering the years 1979 thru
1988 (500mb) and 1979 thru 1985 (1000mb).  

Clustering is based on a correlation procedure.  The first and
second clusters are based on the two runs that are most unlike
each other (the extremes within the ensemble), and other runs
similar (anomaly correlation greater than 0.6) to these are grouped
into these two clusters.  The remaining forecasts are grouped into
up to 4 additional clusters.  Each run can be in only one cluster. 
A cluster may consist of only 1 run.  When the anomaly correlation
between the most unlike runs is greater than 0.6, all the runs are
in cluster 3.

The cluster means are formed from unweighted averages of the
runs.  We use 17 runs rather than the 46 runs (5 cycles) used for
extended-range prediction because at short to medium range the
runs are often grouped into clusters by cycles.  For short and
medium range it makes more sense to cluster one day's runs and
compare these clusters with those made on previous days.  The
cluster calculations are performed on a rectangular area
surrounding the 48 states (25N to 65N and 60W to 130W).  In the
future we plan to make different clusters based on different
areas, generally corresponding to the areas of the maps they are
shown on, and also separately for the eastern and western USA.

The cluster plots are labelled with somewhat cryptic titles that
show which runs are included in each cluster.  The title ends
with two strings of letters and a number.  Each string of letters
represents the runs in one forecast cycle (00Z and the preceding
12Z).  Each letter in the two strings represents one run in the
ensemble: m=T126 MRF, c=T62 MRF, a=AVN, p=positive perturbation
of the preceding control run, n=negative perturbation of the
preceding control run.  Each capitalized letter represents a run
present in this cluster, while lower-case letters are
placeholders for runs not present.  The number following the two
strings is the number of runs included in this cluster.  For
example:

McpNpnPnpnpN aPnpn 5

indicates that this cluster includes the T126 MRF, 3 MRF
perturbations, and one AVN perturbation, making 5 runs in all.

LOCATION OF GRIB DATA FOR INDIVIDUAL ENSEMBLE MEMBERS:

How to access NCEP global ensemble products in grib format:
ftp to:
nic.fb4.noaa.gov
For userid, type:
anonymous
for password, use your full ip-address.
Then cd pub/ens.
There you will find 15 files. The filenames will identify the
variables contained in the files. 
We have:
Height at 1000
          700
          500
          250 mb
Winds at  10 m
          850 mb
          250 mb
Temperature at 2 m
          850 mb
Rel. Humidity  700 mb
Pressure reduced to mean sea level PRMSL
Accum. Precip.
Right now the precip data is for 24 hr non-overlapping periods
for the 00z and 12z runs. There may be problems with the close to
surface variables, too.  These files do not contain in the record
headers the ensemble extension description yet.  Each file is for
one variable, with 17 model runs. We will put out a description
of the files and some software to read the ensemble grib
extension headers in the near future.

The first data record of the file contains the T126 control; the
second is the t62 control; records 3-12 are perturbations from
the 00z cycle (neg-pos pairs); record 13 is the 12Z (previous
day) control and 14-17 are 4 perturbed runs from 12Z. The first
set of 17 data records is for lead time 0 hr (12 hr for the 12Z
runs), the next is for 12 hr, 24 hr etc. out to 10.5 days, after
which it is given only every 24 hrs.  Please give us your
feedback regarding the format of the files.  We are also
interested learning about your experience with the use of the
data.  General information on decoding and unpacking grib data,
along with some software can be found in readme or README files
on:
/
/pub/info
For information and software on how to read grib format files,
using different platforms:
cd /pub/info

NOTES ON INTERPRETATION

It sometimes happens, especially for the AVN cycle, that all the
perturbation fields have the same local shape;  in that case, the
shorter range forecasts (in which the response to the
perturbations is fairly linear) may show the perturbations
grouped in pairs or 5s.  Occasionally the grouping into 2s or 5s
is different in different active regions, for example, western
Atlantic and eastern Pacific, because the signs of each major
feature of each perturbation field may vary at random from one
area to another.  In such cases we may get 3 or 4 clusters which
represent the various combinations of eastern and western
patterns.  For this reason we are planning future cluster
products in which the clustering is done separately for the
eastern and western USA.

If you believe that our ensemble adequately covers all likely
outcomes, then you can take the probability fields as a predicted
probability of the event.  In our experience it is slightly more
likely that the verification will fall outside all the
perturbations than that it will fall between any particular pair
of consecutive perturbation values,  so this assumption is almost
true and the ensemble probabilities are slightly (less than 10%)
closer to 50% than they should be.  

Since all the spaghetti diagrams are for 12z,  you can look at
the 00hr AVN fields as verification, and the 00hr AVN
perturbations show the initial anomaly amplitudes.
 

REFERENCES:

On the generation of the NCEP ensemble, see Toth and Kalnay,1993,
BAMS, Vol. 74, p. 2317-2330.) and NCEP Office Note 407.  On the
use of ensemble forecasts at NCEP, see Tracton and Kalnay, 1993
(Weather and Forecasting, Vol.  8, 379-398).