# Presentation: 391211

## Historical QPF VSDB files:  Streamlining/Consolidation

- A supplement to _[http://www.emc.ncep.noaa.gov/mmb/ylin/pcpverif/scores/](http://www.emc.ncep.noaa.gov/mmb/ylin/pcpverif/scores/)_
- Last updated 9 Apr 2008

## Old Verification System

| Model  | Began on | Verif. Grid |
| --- | --- | --- |
| Eta/NAM | 1992/06/01 | G190 |
| NGM | 1992/06/01 | G104 |
| AVN | 1992/12/04 | G126 |

- QPF verification started in 1992.  In the beginning verification is only done on model’s native grid.

## Timeline of the Precip Verif System

- 1996/04/01: Mike Baldwin began verifying Eta QPF on 9 sub-regions of G190
- 1999/03/22: began verifying all models on G211 and G212
- 2002/02/01: *New* verification system began
- 2006/10/06: *Old* verification completely discontinued (only NAM and NGM remained at that time)
- 2007/12/12: Precipitation verification became operational.  G190 verif for Eta/NAM and G104 verif for NGM discontinued.

## Verif of AVN/GFS in Old System

- 1992/12/04: Began verif on G126
- 1999/03/22: G211/212 verif added
- 2000/01/26: G126 verif replaced by “G255” verif (likely the T170 grid, following the GFS upgrade from T126 to T170)
- 2002/02/01: Verif in new system began
- 2005/01/26: Verif in old system discontinued

## New Verification System

- More streamlined and modular, used for both 3-hourly and 24-hourly verifications.  Ease of adding new models to verify.
- For 24h verifications, use the CPC 1/8 deg daily gauge analysis (copygb’d to verif grid), rather than box-averaging raw gauge reports to verif grid (old approach).

## New analysis on 80km grid

- Old analysis on 80km grid

- New analysis on 40km grid

- Old analysis on 40km grid

**Notes:**
Old analysis: raw gauge data box-averaged to various analysis grids
CPC 1/8 deg anl: PRISM/least-squares scheme

## New(left) vs. old (right) scores 

**Notes:**
Highest new Eta score: about 0.59 vs. 0.56 in old.

## Old vs. New, Eta Seasonal, 1999/01 – 2003/09

- 24h (solid) and 48h (dashed) forecasts on Grid 211

**Notes:**
We now use the new verif system for most purposes such as monitoring current ops models, evaluation parallel runs, annual comparison of NCEP models with half a dozen or so int’l models. 
In this plot, new tht score up to 0.03 higher than old score.

## Effect of Different Verif Grids: #1

- Annual scores, 24h forecast.
- Red: G126
- Green: G211
- Blue: G255 (likely the T170 grid)

- Old GFS Eq Threat Scores on G126/211/255

## Effect of Different Verif Grids: #2

- Annual scores, 24h forecast.
**NAM:**
-   - Red: G190
-   - Blue: G211
**NGM:**
-   - Green: G104
-   - Magenta: G211

- Old NAM and NGM Eq. Threat Scores

## Effect of Different Verif Grids: #3

- Annual scores, 24h forecast.
- Red: G190
- Blue: G211
- Green: G212
- Magenta: G218

- New NAM Eq. Threat Scores

## Effect of Different Verif Grids: #4

- 24h forecast
- 2002/02-2005/12
- Red: G190
- Blue: G211
- Green: G212
- Magenta: G218

- New NAM Eq. Scores by threshold

## Effect of Different Verif Grids: #5

| Threshold (inch/day) | G190 | G211 |     diff:
G211-G190
     G190 |
| --- | --- | --- | --- |
| 0.25 | 0.44645 | 0.44814 | 0.0017 |
| 1.00 | 0.29219 | 0.30500 | 0.0438
 |

- NAM 2006 scores (new), G190 *vs. *G211

## Merging of VSDB

- Merging old and new scores for NAM, NGM and GFS (for long-term performance review).
- Filled in ‘new’ VSDB prior to Feb 2002 with old VSDB
- Renamed GFS’s G126 stats, NAM’s G190 and NGM’s G104 stats to ‘G211’ prior to 1999/03/22 (when G211 verif started), and use this ‘pseudo-G211 verif’, going back to Dec 1992, for historical records
- Eliminated the “specialty” G104 and G190 verification grids when precip verification became operational in Dec 2007

## A sed saga: contents ofgfs-nam-ngm-vsdb.199202-200704.vsdb.tar.gz

- 20020201–20041130: new eta_$yyyymmdd.vsdb files are renamed to nam_$yyyymmdd.vsdb, and inside the files, ‘ETA‘ replaced by ‘NAM‘
- 19921204-20001231: old avn_$yyyymmdd.vsdb files are renamed to gfs_$yyyymmdd.vsdb, and inside the files, ‘AVN‘ replaced by ‘GFS ‘
- 19920601-20020131: old eta_$yyyymmdd.vsdb files are renamed to nam_$yyyymmdd.vsdb, and inside the files, ‘ETA ‘ replaced by ‘NAM‘
- Empty files are not converted (checked with “if [ -s $file.vsdb ];”)
- 19920601-20020131: for gfs/nam/ngm, copy old vsdb files into new vsdb database, after changing “MB_PCP” to “MC_PCP”.  Also, for the subset of vsdbs that are prior to 19990321, rename GFS’s G126, NAM’s G190 and NGM’s G104 to G211.

## Keep in Mind ...

- Scores under the “new” system are slightly higher than in the “old” system
- Due to the change to the “new” system in Feb 2002, annual time series tend to show a jump in 2002
- Scores verified on a finer grid tend to be lower than that on a coarser grid (though bias is similar)