# Presentation: 809271

## (NOAA Operational Model Archive and Distribution System)

**Real Time NOMADS**

- Jordan C. Alpert  _[jordan.alpert@noaa.](mailto:jordan.alpert@noaa.gov)__[gov](mailto:jordan.alpert@noaa.gov)_
- Jun Wang                  _[jun.wang@naa.gov](mailto:jun.wang@naa.gov)_
- Glenn Rutledge	glenn.rutledge@noaa.gov       NOAATECH  10/23//03

**where the nation’s climate and weather services begin**

**(NOAA Operational Model Archive and Distribution System)**

## NOMADS is an agreement between agencies who participate to have common ...

- Data and observation distribution software, format independent and description  methodology,
- Documentation and organization framework,
- Forum to plan and organize
- Funding avenue to direct $.
- ... for university, federal agencies and organizations, a mechanism to obtain support for the dissemination of their data sets.

**Notes:**
There are many participants in NOMADS – See the talk from NCDC. 

## Participants

* *

*Participants*
*U.S.-*
*NOAA, DOE, NASA,*
*NCAR, Unidata, COLA,*
*OPeNDAP Project,  *
*& Several Universities*
*International- *
*CEOS*
*NERC*
*ESG*
*UK BADC *
*BOM / *
*Dept. of State via CAP*

**Notes:**
Here is a slide from a NOMADS talk on the participants

## NOMADS and the NCEP Real Time Component

- Wider information dissemination through web pages and other avenues.
- Framework to create specialized and value added products needs access to weather data.
- Facilitate communication among commercial, public, research community and federal agencies.
- NOMADS is a system for participants to interact with others using common protocols and formats.  Existing differences in details are mollified by software.

*A solution for Recommendation 5 of “Fair Weather: *
*Effective Partnership in Weather and Climate Services”*

## RT-NOMADS at NCEP

- Distributed Ocean Data Server  - OPEN_DAP (DODS) based on metadata file descriptors and XML.
- Web based ftp, the NCEP _[ftp2u](ftp://ftp2u/)_ client and “Great Displays” (pdisp), a grid analysis and display client.
- Model initialization ingredients, “The minimum set” for model rerun/retrospective analysis ready observations & restart files.
- The Real Time NCEP Operational model run history.

*Consists of*

## RT-NOMADS is a service for science, development and public access.

*To distribute NCEP’s Operational data set to researchers and the public.  *
*To provide server software to NOAA projects who need to serve their data through web services.*
- NCEP NOMADS servers are the intermediary between NCEP operational computers and NCDC NOMADS archive servers.

## Tools for Users on the server

- Pare down large file sizes of high resolution  initialization and model results.
- (re-) Group different data sets to create needed products – such as initialization files for model development.
- Subset the data sets in parameter space
- Subset the data sets in physical space
-  (subset the grid in 4 dimensions)

## NCEP Real-Time NOMADS Server

- Real time.
- Operational Suite of Model products.
- Forecast information for model comparison.
- Operational initialization and observations.
- Maintenance of codes.
- Utilities for data manipulation and grid transformation & sub-setting.
- Documentation.
- Serves as transfer point to NCDC for NOMADS archive of reruns and retrospectives.

## The Server – Client distributed relationship:A Common Sense Approach to Data Distribution

- _[ftp2u](ftp://ftp2u/)_, Great Displays, and GDS allow sub-setting of data by parameter, by 3D region, and by time.
- Allows for results with low band width!  Eg., NASA/GSFC takes 1 hour to ftp entire GFS (AVN) forecasts or 13 seconds to download their  desired fields (_[ftp2u](ftp://ftp2u/)_).
- “Great Displays” (pdisp) A display program operating from the same database as ftp2u to  display any part of the NCEP data base by time, region, vertical level and variable.

*Examples of Clients :*

**Notes:**
NASA and NOAA have fast lines between them!

## The GrADS-DODS  Server (GDS)

-  NOMADS participants serve their data sets through a client-server relationship, that is, the data sets are internet ready and the display is done by the client.
-  GDS combines both GrADS, a freeware client (from COLA) and DODS (OPeN-DAP) server to compress and exchange data in many formats with http.
**This means that data appears to the user or client application as a **_**local file**_**!**
*                     **Like Network file system over the web!*
- DODS requests are made by many freeware and commercial high level language clients.
- Simple http queries to the DODS server can create value added products.

## ... A  GrADS client example....

- Sdfopen http://nomad2:9090/dods/reanalysis-2/pgb/hgt
- t = 1
- 'set gxout shaded'
- while (t <= 1068)
-   'set t 't
-   'q time'
-   date=subwrd(result,3)
-   'draw title hgtprs 'date' 't
-   'd hgtprs'
-   '/home/wd23ja/grads/lib/cbarn.gs'
-   'swap'
-   t = t + 1
- endwhile

## ... Using http to extract data from the server:

- http://140.90.198.156:9090/dods/eta/eta20030527/eta_00z.ascii?tmpprs[0:0][0:0][130:130] [290:290]
- the order of the square bracked values is:
-  [time][level][lat][lon]
-  where lat is measured from the south pole (0) to NP (180)
- In units represented by the metadata descriptor file.
- (Use a “wget” and the URL in cron or a cgi-bin script provides needed values.)

## MATLAB: A Commerical Client

- server=‘nomad2:9090’;
- directory=‘/dods/gdas/rotating/’;  fext=‘ascii’;
- model=‘gdas’;  datestr=‘YYYYMMDDHH’;  varname=‘hgt’;
- varlevs=‘1’;
- Arrc=[‘[‘,t1,’:’,t2,’][‘,nlev1,’:’,nlev2,’][‘,ilat1,’:’,ilat2,’][‘,ilon1,’:’,ilon2,’]’];
- urlstring=[http://’,server,directory,model,datestr,’.’,fext,’?’,varname,arrc];
- url = java.net.URL(urlstring);    % connect to web page
- Try
-      stream = openStream(url)
-      ireader = java.io.InputStreamReader(stream);
-      breader = java.io.BufferedReader(ireader);
- Line = readLine(breader);       disp(line);

## NOMADS Data Sets at NCEP 

- Real Time Operational model data sets from GFS, and ETA from present to one month ago.
- Attention to accumulation variable differences and high and low resolution.
- Analysis pgb, restart files.
- Recent (“real time”) climate runs, ie., RSM.
- Observations (soon).
- Reanalysis-2 (and 1)
- AMIP type runs (Suru’s finest runs eg., 28 and 64 levels 1978-2002), sudden warming, ....

*Metadata descriptor files describe all aspects of the data sets and are created uniquely from the headers of GRIB files.  Data sets at NCEP:*

## .... Data Sets (continued)

- NOAAPORT Operations.
- Continuation of NCEP real time model component.
- NCEP model initialization ingredients, “The minimum set” for model rerun/retrospective analysis ready observations & restart files.

*Archives** at NCDC*

## Model Data Req’s - NWP and GCM’s 

- GCM- Global Climate Model
- Reanal- N. Am. Regional Reanalysis
- NWP- Numerical Weather Prediction

- * 5-year rotating NWP archive

- Alpert & Rutledge 6/03

## RT NOMADS Server at NCEP

- Two 1TB RAIDZONE servers
- _[http://nomad2.ncep.noaa.gov](http://nomad2.ncep.noaa.gov/)_
- _[http://nomad1.ncep.noaa.gov](http://nomad1.ncep.noaa.gov/)__       (backup)_
- Documentation web page under construction

## Value added Retailers who make value added products can use NOMADS GDS to get the meteorological data they need without downloading entire files.
Users (read forecasters) of NDFD can create their own products using GDS server accessing only data they need.
GDS reduces the bandwidth needed to create products in weather service operations.
For internet-2 bandwidth, servers at Regional Centers can distribute data to WFO’s for their operations.

- Users (read forecasters) of NDFD can create their own products using GDS server accessing only data they need.
- GDS reduces the bandwidth needed to create products in weather service operations.
- For internet-2 bandwidth, servers at Regional Centers can distribute data to WFO’s for their operations.

- Value Added Products and the
- National Digital Forecast Database

## Demonstration