NCEP Grid to Grid Verification System

Binbin Zhou

NOAA/NWS/NCEP/EMC

July 17, 2008

**Abstract**

This document briefly describes the NCEP grid to grid (g2g) verification
system. The g2g verification system is an extension from the NCEP grid
to station system of (grid2obs, see
<http://www.emc.ncep.noaa.gov/mmb/papers/shafran/gridtob.pdf> ) and aims
to verify forecast model grid output against another verification grid
data such as analysis or satellite data. At current version, GRIB1 grid
data format is required. The g2g system has following features: (1) with
help of copygb, an NCEP's data interpolation tool between different grid
resolutions, model output grid can be different from the grid of
verification dataset, (2) verification region is extended from mesoscale
to global scale so that waving (filtering) and latitude adjustment are
considered, (3) climatologic anomly and tendency can be verified, and
(4) user can verify any sub-regions by specifying their lowest-left and
upper-right points. Like grid2obs, the g2g system also uses a user
control file to specify the verification times, statistic types,
parameters and levels. The output of the g2g is VSDB ascii format files.
As instruction, some application examples are discussed in this report.

**1. Introduction**

The NCEP grid2grid (g2g) System constructed the VSDB (Verification
Statistics Data Base) file by verifying gridded forecast against gridded
observations/analysis data according to the requests from a user-defined
control file. The framework of g2g is similar to that of the NCEP
gridtobs (g2o) which is to verify gridded forecasts against stationed
observation data. See Figure 1

Note: when verifying gridded forecast against another gridded forecast,
the functionalities of the verification could be defined as (1)
Comparing/diagnosing different model forecasts (2) forecasts consistence
for different lead time.

***Two phases in g2g system*:**

Phase I, prepare the GRIB files for both forecasts and verification data
by prepg2g.sh script.

Phase II, compute statistical types such as L1L2, FHO, etc and save them
into the VSDB files

The prepg2g.sh does following jobs according to the user-defined control
file:

(1) Configuring/checking forecast file and verified analysis/another
    forecast file, including their

storing directory paths, names, times, and generating one-to-one pairs
of forecast/analysis or

forecast/forecast files for each forecast lead time

\(2\) Grabbing all needed forecast and verified GRIB files and thinning
them

First look at the variables defined in the control file, then grab data
only for these defined variables

from both forecast and verified GRIB files, storing all of them (all
variables and all leading

forecast) into one forecast GRIB file and one verified GRIB file

\(3\) Reconstructing user-defined control file by adding more
information on forecast and verified data time pairs, including both
regular variables and time tendency variables (see later part) and
storing all settings defined in user-defined control file plus pair
information in a new control files g2g control

Phase 2 is actual work done by a Fortran program grid2grid. The
grid2grid was developed by modifying gridtobs program(Geoff DiMego). The
general structure is same but adding more subroutines and functions to
deal with grid configurations, FHO, tendency computation, climate
anomly, global wave

groups, etc.

The grid2grid program first reads all parameters from g2g control file
and both the fcst data and verified data from their thinned GRIB files,
then calculates SL1L2, VL1L2 and FHO, writing out the results into a
VSDB file. Files regions and grid#104 are for setting the sub-regions.

In summary, the general features in addition to those of g2o, g2g has
more including:

\(1\) Can be used on global and regional models verifications

\(2\) All variables (scalars or vectors) in GRIB can be requested

\(3\) Has time tendency (3, 6, 12, 24 hours) verification

\(4\) FHO or AFHO (anomly) works on both regular variables and time
tendency variables

\(5\) User can specify all of 29 sub-regions already defined in
gridtobs, even in case of global forecast

\(6\) User can specify any rectangular sub-region within the grid domain
by two geographic points

Plus: northern and southern hemispheres, tropic regions, etc

(7) Can request climatologic anomly on some variables at some levels

(8) In global grids, variables can be selected by different wave group
    numbers (e.g. 1-3, 4-9, etc)

**2. Description of user-defined control file**

Although the g2g verification package is adopted from g2o and very
flexible to use, its requesting grammar in the user control file must be
followed. This section will introduce how to set

the requests in the control files

For users, all needed is defining their own control file and make sure
the forecast and verified data files are located proper locations that
can be accessed by prepg2g.sh.

*Part I. Syntax of user-defined control file*

Similar to g2o, users specify their parameters also with 9 headers in
the user-defined control file which has same structure with that of g2o,
but there are some difference to deal with FHO/AFHO, tendency, anomly,
and wave filtering. When g2g VSDB file is generated, the NCEP FVS
utility can be used to create various verification statistics. The 9
headers are

Header 1: Version

Header 2: Model names

Header 3: Forecast times

Header 4: Verified times

Header 5: Verified data source

Header 6: Grid/Regions

Header 7: Statistics types

Header 8: Variables

Header 9: Levels

Let's briefly explain each header in the control file

*Header 1*: version information, similar to g2o, such as V01

*Header 2*: model name or names similar to g2o, such as ETA-212,
NAM-221, GFS-2, etc,

the number after "-" is the grid# which can be omitted, the string after
"-" can be any others

such as 32km, etc. Like g2o, multiple model names can be specified one
by one, but for

deterministic forecast verification, only one model name is recommended.

Example

1 NAM-32km

*Header 3* and *Header 4* are combined together to match the forecast
time and verified times

The verification can be classified as following 2 types (see Figure 2)

Case (i) One verification is used to verify previous multiple cycles of
forecasts

Case (ii) One cycle forecast is verified by future verification data at
multiple lead times

For case (i), header 3 and 4 should be specified as following format,
for example,

4 12 ..........Heade3

18

24

30

1 2005031218 .........Header 4

where 2005031218 is the verified data time at 18Z, March, 12^th^, 2005,
which will be used to verify

the four cycles of forecast: 12, 18, 24, 30 hours before the 2005031218.

In the Header 3, the number 4 the total forecast times specified. User
can specify any number of forecast times as long as the largest forecast
time not greater than the longest forecast time

In Header 4, only one time is specified. No more than one can be
specified in this case

Fore case (ii), header 3 and 4 should be specified as following:

1 2005031206 ........Header 3

3 12 ........Header 4

18

24

where 20005031206 is the forecast cycle time, at 06Z, March 12^th^,
2005. Its 12, 18, 24 hour forecasts (lead times) will be verified by
three verification data. 3 in header 4 is the total number of forecast
times to be verified, and only one cycle time should be specified in the
header 3 in this case.

If a user wants to compare the forecasts from two models, this work also
can be done by arranging one of forecasts as "observation" and one as
"forecast", but don't forget, the two model outputs must be same grid
number. If not, copygb can be used to convert one of them to another.

According to the above user specified header 3 and header 4, prepg2g.sh
will automatically searches and matches each pair of forecast and
verification data. If not found in the file system, a warning will be
given and skip them when grabbing the data. The time information of each
pair will be stored in the g2g control file for grid2grid program. The
time information is in the string formatted as "YYYYMMDDHHFF". Users
don't worry about the g2g control file. It is handled by program. For
user, the only useful information is the tendency verification.

The tendency (3, 6, 12, 24hr) verification needs previous 3hr, 6hr,
12hr, 24hr GRIB forecast files. After finding one forecast and
verification pair, prepg2g.sh will further search all GRIB files for 3,
6,12,and 24 hr tendency and store the time-matching information in the
g2g control file as well. If not find, FF is marked with "NN". Then in
the Phase II, grid2grid will automatically skip the tendency computation
of these times. If no tendency is requested, these information is still
listed there, but will be not read by the g2g program.

*Header 5:* The verified data source, similar to g2o. Please note that
both forecast and verified data file should be in the same GRIB format,
otherwise, use copygb (use bicubic interpolation, i.e. flag "-i1,1")

to make them in same GRIB format. Example:

1 GDAS/2

Indicating the verified data from GDAS, grid number is 3

*Header 6*: Same as g2o, but user can specify additional regions besides
those defined in go2:

The format for header 6 is "Gnnn" or "Gnnn/RRR, where nnn is grid
number, RRR is three-character sub-region name defined in the file
"regions". "Gnnn" without "/RRR" means whole grid domain. For example,
G212 means whole grid 212 domain, G212/NWC means northern west coast.
For global, G2, G3 G29, G30. The additional regions can be northern
hemisphere (such as "G2/NHM"), or southern hemisphere (such as
"G2/SHM"), which are not defined in file "regions" . Since 29
sub-regions in regions file are defined in grid 104, please note that,
if a grid domain smaller than, or only a part of grid 104 domain, the
effective area is the intersection of defined grid domain/sub-region and
grid 104 domain.

Users can define their own rectangular region by setting 2 left-lower
and right-upper points in header 6. The syntax is "Gnnn/RRR Lon1 Lat1
Lon2 Lat2" Where RRR is any string defined by the user (less than 20
characters). "Lon1 Lat1" is the left-lower point's longitude and
latitude, and "Lon2 Lat2", the right-upper point's longitude and
latitude, where longitude should be in eastward. (negative longitude is
not permitted)

In Figure 3, two user-defined regions are specified. If the two pints
are both within the domain, then all grid data within the region will be
taken into account, however, if one point (or even two points) is (are)
outside of grid domain, only intersection part between grid domain and
user-defined region is considered.

If the points are specified in such way that their longitudes are
exactly same, then the region between the first point's latitude and
second point's latitude are defined. This specification can be used to
verify tropical area forecast along the global longitude.

Please note, for global grid, the user defined region is not permitted
across 0 degree longitude. In the next version, this problem will be
solved.

Example

5 G221

G221/NEC

G221/HWI

G221/USR1 220 35 240 45

G221/USR2 210 20 230 60

This example shows that, 5 regions are specified for grid 221. The first
is for whole domain, the 2^nd^ and 3^rd^ are northern east coast and
Hawaii, the last 2 are user-defined regions

*Header 7*: Specification of statistic types, similar to g2o, current
version has

\(1\) SL1L2 \-- L1L2 for scalar variables

\(2\) VL1L2 \-- L1L2 for vector variables

\(3\) FHO \-- Forecast rate (F), Observation rate (O) and Hit rate (H)

\(4\) SAL1L2 -- Anomly L1L2

\(5\) VAL1L2 -- Anomly VL1L2

\(6\) AFHO \-- Anomly FHO

More statistic types will be added in the later versions upon request

Example:

4 FHO

SL1L2

VL1L2

SAL1L2

Another example

6\. SL1L2

VL1L2

SAL1L2

VAL1L2

FHO

AFHO

*Header 8*: Specification of requested variables.

All variables can be specified as long as they are defined in the GRIB
data file. The string names are recommended same as those defined in the
GRIB Office Note 388 Table 2, but without space. But the type of
requested variables should coordinated with Header 7. The formats of
Header 8 for different type of requests are

(1) SL1L2, VL1L2 , SAL1L2, VAL1L2

VName kpds5 kpds6 kpds7

Where VName is the string name defined by user, and suggested less than
5 char length but no limitation. The kpds5,6,7 are the kpds number
defined for each variable in GRIB (Office Note 388).

(2) SL1L2, VL1L2 with **tendency** VNAME_TND/tt kpds5 kpds6 kpds7

Where tt is either of 3, 6, 12, 24. The \_TND must be specified

Note: (SAL1L2 and VAL1L2 can not request anomly)

(3) SL1L2, VL1L2, SAL1L2 , VAL1L2 with selected **wave** groups

VNAME_WV/w1-w2 kpds5 kpds6 kpds7

Where w1-w2 is number of waves. For instance, 1-3, 4-9, 10-20, 0-20,
etc. The \_WV must be specified

Anomly requests for scalars and vectors are identified by SAL1L2 and
VAL1L2 in Header 7, but it

Should be noticed that, not all variables can be requested to do anomly
computation. Only following variables at certain pressure levels can do.

Hight ( 7, 100, 0) at 1000, 700, 500, 250mb

PSL ( 2, 102, 0)

T (11, 100, 0) at 850, 500, 250mb

T2m (11, 105, 2) at 2m

T2max (15, 105, 2) at 2m

T2min (16, 105, 2) at 2m

U (33, 100, 0) at 850, 500, 250mb

V (34, 100, 0) at 850, 500, 250mb

U10m (33, 105, 10) at 10m

V10m (34, 105, 10) at 10m

The program can automatically find which one can be requested for
anomly. For example, Header 7 and

8 appear like following:

2 SL1L2

SAL1L2

3 TMP 11 105 2

TMP_TND/12 11 105 2

RH 52 105 2

RH_WV/1-3 52 105 2

This request says, SL1L2 for 2m Temperature , for 2m temperature 12hr
tendency change , for 2m RH and for 1-3 waved 2m RH. SAL1L2 for 2m
Temperature anomly only, but not for 2m Temperature tendency, not for 2m
RH, and not for waved 2m RH because anomly has no tendency request and
RH can not request anomly. This selection will be done by g2g program
automatically. Users don't worry about it.

Another example:

1 SL1L2

VL1L2

4 TMP 11 100 0

TMP_TND/12 11 105 2

PRES 1 1 0

WIND 32 100 0

This example shows that, 4 variables are specified: temperature on
pressure, 12 hr temperature tendency at 2 m above the ground, the
surface pressure, and the wind speed at pressure levels (kpds6=100).
Please note:

\(a\) for variables not on the pressure, if at specific level above the
ground, kpds7 must be specified

\(b\) for variables on the multiple pressure levels, kpds7 always set to
0 (this setting is same as that in the

GRIB file), but which pressure levels user wants to verify will be set
in the header 9 (see later)

\(c\) **The vector variables must be put in the last line of the header
8** (WIND in this example), which

is same as g2o. When specify wind speed, preg2g.sh will automatically
grab its U and V

components into the thinned GRIB file

\(4\) For FHO or anomly FHO (AFHO), additional threshold information is
needed. These threshold values are put following kpds7 as following
syntax.

(a) For non-anomly variables

VName kpds5 kpds6 kpds7 FHOop t1 t2 t3 .....

VName_TND/tt kpds5 kpds6 kpds7 FHOop t1 t2 t3 .....

VName_WV/w1-w2 kpds5 kpds6 kpds7 FHOop t1 t2 t3 .....

Where op is an operator "\<", or "\<", or "="

(b) For anomly aviables (no tendency)

VName kpds5 kpds6 kpds7 FHOop t1 t2 t3 .....

VName_WV/w1-w2 kpds5 kpds6 kpds7 FHOop t1 t2 t3 .....

For example:

2 FHO

AFHO

2 TMP 11 105 2 FHO\> 270.0 280.0 290.0

TMP 11 105 2 AFHO\> 270.0 280.0 290.0

where 270.0, 280.0 and 290.0 are three thresholds for 2m temperature and
2m temperature anomly . "FHO\>" is a string means to find FHO larger
than these thresholds. For less than case, use string "FHO\<". The "FHO"
and "\>" or "\<" can not be separated by any space. In current version,
up to 20 thresholds in total can be specified. The same thing is for
anomly FHO (AFHO).

For tendency case, user needs to denote "increase" or "decrease" and
"increase amount" larger/less than certain threshold, or "decrease
amount" larger/less than certain threshold. All of these 4 cases can be
set as followings, for example,

TMP_TND/12 11 105 2 FHO\> 2.0- 4.0-

TMP_TND/12 11 105 2 FHO\> 2.0+ 4.0+

TMP_TND/12 11 105 2 FHO\< 2.0- 4.0-

TMP_TND/12 11 105 2 FHO\< 2.0+ 4.0-

The first line is to verify FHO for 12hr 2m-temperature's decrease
larger than 2 C

If TMP_TND/12 \> - 2 (TMP\_ TND /12 \> 0 or \| TMP\_ TND /12 \| \< 2 ),
false, otherwise true

The second line is to verify FHO for 12hr-2m temperature's increase
larger than 2 C

If TMP_TND/12 \> 2 , true, otherwise false

The third line is to verify FHO for 12hr 2m-temperature's decrease less
than 2 C

If TMP_TND/12 \> - 2 (TMP\_ TND /12 \> 0 or \| TMP\_ TND /12 \| \< 2 ),
true, otherwise false

The fourth line is to verify FHO for 12hr 2m-temperature's increase less
than 2 C

If TMP_TND/12 \> 2 (TMP_TND \< 0, or TMP_TND \< 2 true, otherwise false)

For tendency case, "+" or "-" can not be omitted from each threshold and
no space between the threshold and "+" or "-", otherwise no output in
the VSDB file for this variable

Note:

The coordination rule of Header 7 and Header 8 is

\(1\) SL1L2: Scalars, waved scalars, scalar tendency

\(2\) VL1L2: Vectors, waved vectors, vector tendency

\(3\) SAL1L2: Scalars, waved scalars

\(4\) VAL1L2: Vectors, waved vectors

\(5\) FHO: Scalars and vectors, waved scalars and vectors, scalar
tendency and vector tendency

\(6\) AFHO: Scalars and vectors, waved scalars and vectors

*Header 9*: Pressure or sigma levels specification for all of those
variables specified with kpds6=100 (pressure) or kpds6=107 (sigma). The
values should be among the pressure levels already defined in the GRIB
file. For example:

5 P1000 or 3 S10000

P900 S9975

P850 S9915

P800

P750

The "P" or "S" must be the first letter followed by a pressure or sigma
number

In case that the there is no variable is verified on a pressure level,
it is recommended al least one pressure put there, such as just put 1
P1000 at last line of control file.

Other level specifications depend on kdps7 in header 8:

In header 8, if a variable's kpds6 is specified as 105 and kpds7 as nnn
(such as 500) , then in the VSDB file, the level will be "Hnnn"

If a variable's kpds6 is specified as 102 and kpds7 as 0, then in the
VSDB file, the level will be "MSL",

If a variable's kpds6 is specified as 200 and kpds7 as 0, then in the
VSDB file, the level will be "ATMOS"

If a variable's kpds6 is specified as 1 and kpds7 as 0, then in the VSDB
file, the level will be "SFC"

If a variable's kpds6 is specified as 2 and kpds7 as 0, then in the VSDB
file, the level will be "CLDBS"

If a variable's kpds6 is specified as 7 and kpds7 as 0, then in the VSDB
file, the level will be "TROP"

If a variable's kpds6 is specified as 4 and kpds7 as 0, then in the VSDB
file, the level will be "0DEG"

**An complete example of user-defined control file:**

(no empty line between headers)

V01 10

1 GFS-2

4 12

18

36

48

1 2005032806

1 GDAS/2

5 G2

G2/NAK

G2/NWC

G2/NEC

G2/USR 220.0 23.5 225.0 30.0

6 SL1L2

VL1L2

SAL1L2

VAL1L2

FHO

AFHO

14 PRES 1 1 0

PRES_WV/4-9 1 1 0

PRES_TND/24 1 1 0

PRES_TND/24 1 1 0 FHO\> 500.0+ 1000.0+

HGT 7 100 0

TMP 11 100 0

TMP 11 105 2

TMP 11 105 2 AFHO\< 273.0 278.0

TMP_TND/12 11 105 2 FHO\> 1.0- 1.5- 2.0- 2.5- 3.0-

TMP_WV 11 100 0

RH 52 105 2

U_WV/0-3 33 100 0 AFHO\> 10.0 15.0 20.0 25.0 30.0

WIND 32 105 10 FHO\> 10.0 20.0 30.0

WIND 32 100 0

6 P1000

P850

P700

P500

P300

P250

**3. Description of grid2grid**

After running prepg2g.sh which generates thinned forecast and verified
GRIB files and g2g control file, then run grid2grid executable file
grid2grid.x. g2g control file is as same as user-defined control except
replacing the forecast time and verified time with matching pairs of
forecast and verification times in form of YYYYMMDDHH. If no data
available, HH will be NN, so that grid2grid can automatically skip that
pair.

The grid2grid first read forecast and verification time matching pairs
and variable names from g2g control file, then get the forecast and
verification data from the thinned grib files. According to the grid
number and sub-region specifications, find the grid pints within these
sub-regions, do the computations of SL1L2, VL1L2, and FHO at these
points and put the results in a VSDB files

***Code and script description**:*

Current version of prepg2g.sh is in /meso/save/wx20bz/grid2grid/exe.
After finish running prepg2g.sh,

one thinned forecast grib file and its index file, one thinned observed
grib file and its index file are generated. If tendency requests are
set, the correspondent tendency forecast and observed thinned grib files
as well as their index file will also be generated. The next step is run
grid2grid Fortran program, which is done by running the another script
run_g2g.sh in /meso/save/wx20bz/grid2grid/exe. After finish

run_g2g.sh, a VSDB file will be generated in the working directory.

The grid2grid Fortran source code is in
/meso/save/wx20bz/grid2grid/sorc.2007215, including a makefile.

All libraries needed for compiling can be found from that makefile. Just
type make to compile it on NCEP

CCS.

When execute prepg2g.sh, please make sure you are correctly set your own
user-defined control file, and correct set the paths and file name for
both forecast and verification grib files. The VSDB result file will be
stored in the working directory.

There are 4 "open-close " buttons for 4 tendency. If only one tendency
(e.g. 12hr) in the user-control file, then only open correspondent one
tendency button. If no tendency, then close all tendency buttons.

There is a sample script file can be referenced:
/meso/save/wx20bz/grid2grid/nam/g2g_run_sample.sh.

This file will help you to create a user control file, setting forecast
file path and observation file path, run prepg2g.sh and run_g2g.sh
scripts.