# Presentation: 015747

## NOAA’sNational Weather ServiceNDFD Data Flow, Format, and Statistics

***Tim Boyer******Meteorological Development Laboratory****National Digital Forecast Database Technical Workshop*
*August 13, 2003*

## NWS Digital Forecasting

***Overview***

** ****NDFD data flow**
**Timeliness of data**
** ****Dissemination of NDFD data**
** ****Data format details**
** ****Statistics tracked in the NDFD**

## NDFD Data Flow

**NDFD System Architecture**

**Replicate  **
**Server**

**Web Farm**

**NCEP/NC**
**(6)**

**AWIPS **
**  ****WAN**

**Regional **
**Headquarters**

**NCF**

**Partners  & Customers**

**Internet**

**TOC**

**FTP**

**NDFD**
**Central Server**

**NCF LAN**

**FTP **
**Server**
**Products**
**(NWSTG)**

**Database**
** ****Server(s)**
**(Future)**

** ****Web Server**
**(Future)**
**HTTP Services**

**Database Push**

**NCDC**

**I-2**

**NWS**
**Net**

**NDFD**
**Stage &**
**Develop**

**Regional **
**Web Farms**

## NDFD Data Flow

**AWIPS WAN**

**Grids (Official Forecast)**

**WFO**
**(120+)**

**NCEP**
**(6)**

**NDFD**
**Central**
**Server**

**NCF**

- Forecast grids are encoded in GRIB2 at WFOs and NCEP
- Grids are decoded and stored to Informix RDBMS
- NDFD mosaics are generated hourly from WFO grids

## Timeliness of NDFD Data

- WFOs send new forecast grids as they are updated
- Gridded forecast data received from WFOs around the clock
- WFO grids decoded at NDFD Central server every 5 minutes and stored to RDBMS
- NDFD mosaics are generated hourly, starting at 10 minutes prior to the top of the hour
- Mosaics are encoded in GRIB2 format and transferred to NWS Telecommunications Gateway (NWSTG).
- Latest hour’s mosaics available for FTP download at NWSTG by 15-20 minutes past the top of the hour.

## Timeliness of NDFD Data

**Thu**

**Fri**

**Sat**

**Sun**

**Mon**

**Tue**

**Wed**

***Thu***

**Day 1**

**Day 2**

**Day 3**

**Day 4**

**Day 5**

**Day 6**

**Day 7**

***Day 8***

**At 1800 UTC every day, drop Day 1 and add new Day 7 **

## Dissemination of NDFD Data

**NCF**

**NWSTG**

**NDFD**
**Central**
**Server**

**FTP Server**

**GRIB2 **
**(Mosaics)**

**PARTNERS &**
**CUSTOMERS**

**Internet**

- Firewall

**GRIB2**

** ****The entire NDFD is updated at **
**   ****the NWSTG every hour**
** ****Latest GRIB2 files can be found at:**
**  ****http://weather.noaa.gov/pub**
** ****Files are arranged in directories**
**  ****according to geographical region**

## Sample FTP Directory (CONUS)

**Current directory is /SL.us008001/ST.expr/DF.gr2/DC.ndfd/AR.conus**
- WARNING
- This is a United States Government (NOAA) computer system, which may be
- accessed and used only for official Government business by authorized
- personnel. Unauthorized access or use of this computer system may
- subject violators to criminal, civil, and/or administrative action.
- Please read the file README.TXT
-   it was last modified on Mon Aug 19 13:36:34 2002 - 345 days ago
- Up to higher level directory
-   ds.maxt.bin              1674 Kb    Tue Jul 30 17:18:00 2002 Binary Executable
-   ds.mint.bin              1397 Kb    Tue Jul 30 17:19:00 2002 Binary Executable
-   ds.pop12.bin             1289 Kb    Tue Jul 30 17:18:00 2002 Binary Executable
-   ds.qpf.bin                634 Kb    Tue Jul 30 17:19:00 2002 Binary Executable
-   ds.sky.bin               3312 Kb    Tue Jul 30 17:17:00 2002 Binary Executable
-   ds.snow.bin               152 Kb    Tue Jul 30 17:19:00 2002 Binary Executable
-   ds.td.bin                7516 Kb    Tue Jul 30 17:19:00 2002 Binary Executable
-   ds.temp.bin              8072 Kb    Tue Jul 30 17:18:00 2002 Binary Executable
-   ds.waveh.bin               81 Kb    Tue Jul 30 17:19:00 2002 Binary Executable
-   ds.wdir.bin              7437 Kb    Tue Jul 30 17:18:00 2002 Binary Executable
-   ds.wspd.bin              3076 Kb    Tue Jul 30 17:18:00 2002 Binary Executable
-   ds.wx.bin                1798 Kb    Tue Jul 30 17:19:00 2002 Binary Executable

## NDFD Data Format (some details)

** ****Each file contains multiple GRIB2 bulletins for one **
**  ****weather element**
** ****Each GRIB2 bulletin begins with a unique WMO header**
** ****Each GRIB2 bulletin contains data for a single weather**
**  ****element for a single valid time**
** ****Temporal resolution is 3 hours for Days 1-3 and 6 hours**
**  ****for Days 4-7**
** ****NDFD data issuance time (YYYYMMDDHH) is encoded **
**  ****in each bulletin**

## NDFD Data Format (some details)

**Special case: Element = “Weather”**

**Weather is handled differently **
**in GRIB2 from other elements**

** ****Weather is a discrete parameter **
**  ****type**
** ****A weather mosaic is defined by a **
**  ****grid and a list of weather “keys”**
** ****In a GRIB2 bulletin, the weather **
**  ****keys are contained within the **
**  ****Local Use section (Section 2) **

## NDFD Weather

| NoWx | No Weather |
| --- | --- |
| A | Hail |
| T | Thunderstorms |
| RW | Rain Showers |
| L | Drizzle |
| ZR | Freezing Rain |
| ZL | Freezing Drizzle |
| S | Snow |
| SW | Snow Showers |
| IP | Ice Pellets |
| F | Fog |
| H | Haze |
| BS | Blowing Snow |
| K | Smoke |
| BD | Blowing Dust |
| FR | Frost |

**Weather Type Codes**

| NoCov | No Coverage/No Prob |
| --- | --- |
| Iso | Isolated |
| Sct | Scattered |
| Num | Numerous |
| Wide | Widespread |
| Ocnl | Occasional |
| SChc | Slight Chance |
| Chc | Chance |
| Lkly | Likely |
| Def | Definite |
| Patchy | Patchy |
| Areas | Areas of |

**Coverage/Probability Codes**

| NoInten | No Intensity |
| --- | --- |
| -- | Very Light |
| - | Light |
| m | Moderate |
| + | Heavy |

**Intensity Codes**

| FL | Frequent Lightning |
| --- | --- |
| GW | Gusty Winds |
| HvyRn | Heavy Rain |
| DmgW | Damaging Winds |
| SmA | Small Hail |
| LgA | Large Hail |
| OLA | Outlying Areas |
| OBO | On Bridges and Overpasses |

**Attribute Codes**

## NDFD Weather

| Sample Weather Key | Meaning |
| --- | --- |
| Sct:SW:-:NoVis | Scattered snow showers |
| Ocnl:R:-:NoVis^Ocnl:S:-:NoVis^SChc:ZR:-:NoVis | Occasional light rain and snow, with a slight chance of freezing rain |
| Wide:FR:-:NoVis:OLA | Widespread frost in outlying areas |
| NoCov:NoWx:NoIntens:NoVis | No Weather |

** **A weather** Subkey **can contain up to 5 weather attributes
** **A weather Key can contain up to 5 Subkeys (usually not more than 3)
** **Gridpoint values are integer indices into the list of weather keys
-  Every mosaic has it’s own unique list of weather keys
-  For more information on weather and encoding details of other NDFD elements, see http://www.nws.noaa.gov/datamgmt/doc/GRIB2_encoding.html

## NDFD Statistics Tracking

**Several different types of statistics are tracked on the NDFD (by WFO):**

** ****Latest receipt times of WFO forecast  grids**
** ****Average forecast update frequency :**
**     ****- daily**
**     ****- weekly**
**     ****- monthly**

## NDFD Statistics Tracking

- By 1800Z each day, WFOs send in a new day’s worth of forecast grids
- (new Day 7).  NDFD statistics are compiled to track 18Z update timeliness:

| WFO | Percentage of Day 7 grids received prior to 18Z |
| --- | --- |
| KABQ | 90.10 |
| KABR | 88.24 |
| KAKQ | 94.12 |
| KALY | 100.00 |
| KAMA | 47.06 |
| KAPX | 100.0 |
| KARX | 100.0 |
| KBGM | 100.0 |
| KBIS | 100.0 |
| KBMX | 100.0 |
| KBOU | 76.34 |

** ****Keep track of daily, weekly,**
**  ****monthly average statistics**
** ****Also track percentages of ALL**
**  ****grids received by 18Z**

## NDFD Statistics Tracking

**WFO Border Consistency Statistics**

** ****Assists forecasters in tracking the coordination their of gridded forecasts**
** ****NDFD processes check (on an hourly basis) the consistency of forecasts across WFO boundaries for an average of adjacent gridpoints**
** ****Terrain effects are taken into account by including average elevational differences along WFO boundaries**
** ****Image as well as tabular output is produced as feedback to forecasters**

## NDFD Statistics Tracking

**NDFD Forecast Verification**** **

** ****Another source of feedback**
**  ****to the forecaster to help**
**  ****improve forecast quality**
** ****Statistics are updated**
**  ****monthly based on previous**
**  ****month’s NDFD forecasts**
** ****Tracks mean absolute error**
**  ****and forecast bias**

## Future Directions

** ****NDFD Database replication to web farms**
** ****Provide simple point forecast products via interactive web queries (i.e., click on an NDFD point)**
**      ****- formatted text forecast out to 7 days**
**      ****- meteograms**
**      ****- NDFD data in XML**
** ****Web Service access to NDFD data via SOAP**
** ****More detailed tracking of NDFD data consistency statistics**